If you aren’t already, you should consider using code version control in Azure Data Factory or Azure Synapse Analytics workspaces. This blog post explains how to enable Azure DevOps. It will also help you meet the pre-requirements (create Azure DevOps project) and commit changes between branches. This is a must-do immediately after you create the service in your development environment.
Azure Synapse Analytics Workspaces and Azure Data Factory are fully integrated with code version control, either Azure DevOps or GitHub. This blog post explains how to enable it.
- Create an Azure DevOps Project
- Configure Azure DevOps in Azure Synapse Analytics (valid for Azure Data Factory)
- Validation of configuration
- Committing changes in Azure DevOps
- Deploying to Publish branch
Create an Azure DevOps Project
To start, create a new project in Azure DevOps.
Then, make the repository private. You can choose between TFS and Git (1) and Agile and Scrum (2). I am a Scrum master, so I always go with Scrum.
Once the project is created, give access to other users and disable capabilities that you won’t use. You always can enable them after later.
If you don’t give your team members access, they won’t be able to use the repository in Azure Synapse Analytics / Azure Data Factory.
Now, you are ready to enable code source version control in Azure Data Factory or Azure Synapse Analytics.
Configure Azure DevOps in Azure Synapse Analytics (valid for Azure Data Factory)
Next, to enable code source control in Azure Synapse Analytics or Azure Data Factory, go to the drop-down at the top of the screen (1) or you can do it from the Manage Hub (2)
The following window will appear so you can select Azure DevOps Git or GitHub.
In this tutorial, I will choose Azure DevOps Git. Choose your account and click next.
You can now fill out the information to set up a code repository.
- Select a name for the repository. It will create one for you.
- Define a collaboration branch (you will create more later). This is where developers can collaborate and merge changes from their branches before they are ready to be deployed into production by using the Publish branch (3).
- The code that is the source of truth and is ready to be deployed to production.
- Import the existing solution in the new code repository.
Finally, click apply.
You need to create a working branch. This is a copy of the master branch and your workspace for applying changes. This helps to create multiple streams of work (developers working on changes at the same time).
Done! You configured code version control in your Azure Synapse Analytics or Azure Data Factory.
Validation of configuration
In Azure Synapse Analytics or Azure Data Factory, you will find different branches.
You can commit single changes or “commit all.”
When you commit, your branch registers the changes.
In your Azure DevOps, you can see the new repository.
In Azure Synapse Analytics Workspaces, there will be more folders than in Azure Data Factory.
Committing changes in Azure DevOps
The next step is to merge your changes back to the collaboration branch. Click to create a pull request.
This will take you to Azure DevOps to merge the changes into the collaboration branch.
Approve and complete the changes (you could have rules, so you are not the one approving and reviewing your own changes!).
If you want to delete your branch because you are not going to apply any more changes, you check the option and click complete merge.
Now your changes are available in the collaboration branch (master) and everybody has access to them.
You can find a summary on the Commits page.
Deploying to Publish branch
If you want to publish a set of changes to the Publish branch before you release them into Production, click publish. The idea is then to use Azure DevOps pipelines to release the changes to a new environment.
It will show you a summary of all the changes to apply. Click OK.
In Azure DevOps, you will see the changes coming through.
Now, the changes in the Publish branch can be released to the next environment (Test, Production, etc.).
Today, you’ve configured code version control for Azure Data Factory / Azure Synapse Analytics Workspaces without writing a single line of code. Everything is fully integrated with Azure.
In upcoming blog posts, we’ll continue to explore Azure Data Services features.
Please follow Tech Talk Corner on Twitter for blog updates, virtual presentations, and more!
If you have any questions, please leave a comment below!