How to avoid (and sometimes fix) duplicate content created by Deployment Pipelines

Generated using Microsoft Designer (first attempt only)

 

If you use Deployment Pipelines, you may sometimes try to move content to a new Workspace, but instead of overwriting the older copy in the destination, running the Pipeline creates a second file in the destination. Power BI thinks they are two different files, but you know they're different versions of the same file.

Below I will describe how to avoid this and also resolve unlinked items, if you already find yourself in this situation. But first it's important to understand how Power BI views files in Deployment Pipelines, which helps you understand the behaviors and resolution to the issue.

How Power BI content gets linked across Deployment Pipeline stages

When publishing from Power BI Desktop to the Service: publishing over a file is based on name only. So if you want to overwrite something as you publish it to the service, ensure they are called the same thing.

The initial file linking when adding Workspaces to a Deployment Pipeline is also based on names. In other words, when a Deployment Pipeline is configured, files called the same thing in linked Workspaces will be assumed to be linked. (I will refer back to this in the Workaround section below.)

When publishing (moving) content from one Workspace to another in the Service via Deployment Pipeline: publishing over a file is based on internal identifier. In other words, two files of the same name may not have the same internal identifier and may be identified by Power BI as unrelated.

Common ways that files get unlinked and result in duplication

If you have multiple copies of the same file in a pre-production Workspace (ex., with dates or version numbers at the end of the name), then rename one of them to the base file name and promote it,  it will create a duplicate in the destination when moved. For example, imagine publishing a few different versions of a report, soliciting feedback of which is the most desirable interface, deleting all but the "best" version, renaming it, and promoting.

• Only the original file that was promoted to the next stage or was present when the pre-production Workspace was linked to the Deployment Pipeline can be migrated to the next stage without creating a duplicate. Takeaway: This original file should never be deleted, and renaming it could create confusion. Publish over this file when you are ready to promote it and don't bother trying to publish other versions/copies of the same report. This is frequently an issue when multiple developers work on the same content in the service and are getting used to collaborating.
  

The internal ID linking a file to where it originated does not survive downloading, so you can't download from Prod and publish that to Dev to re-link the files in Dev and Prod, if they became unlinked. It's a problem that exists in the service only, which must be resolved in the service. What you do from the desktop won't change anything.

Workaround to fix unlinked files

Reread the second paragraph in the top section about the initial Deployment Pipeline configuration. The easiest way to patch is to unmount the Workspace with files that became unlinked (like Development, at least), save the Deployment Pipeline, then mount the Workspace again and save again. This links files with the same names.

With two total development phases, it isn't likely a big deal to unmount and mount the Development Workspace, but if you have to temporarily unlink a QA/UAT, as well, and you had deployment rules set up there, you will probably have to set them up again. (You shouldn't have to unlink Production for any reason.)