Azure Data Factory Linked Services and Parameters

In Azure Data Factory, there are more than 90 pre-built connectors that you can use to connect to your data stores. In this blog post, I’ll show you what Azure Data Factory Linked Services are, how to create them, and how to add parameters. 

Azure Data Factory Linked Services allow you to establish a connection with your data stores. They are connectors you can use while working with assets in data stores. This includes the configuration to access data stores, as well as connection strings and authentication type. 

This blog post applies to Azure Data Factory and Azure Synapse Analytics. 

Relationship between pipeline, activity, dataset, linked services

Image copied from Microsoft documentation 

Create Azure Data Factory Linked Services 

First, go to the Manage Hub. Click on Linked Services and create a new one. 

Manage Hub create new Linked Service

Second, you can see the different categories and connectors that you can use. For this example, I’m using Azure SQL Databases. Click continue. 

Select category

Next, you’ll be taken to the main configuration page for the linked service. Each connector requires different configuration parameters (like connection strings or a file path) while some of them are generic for all the parameters (like name, description, annotations, parameters) 

New linked service configuration
  1. Define a name for your linked service 
  1. Add a description 
  1. Select the Azure Data Factory Integration Runtime to use. You can leave the default one for Azure Linked Services 
  1. Use Azure Data Factory and Key Vault  (recommended) or define the connection string and credentials (not recommended
  1. Use Azure Data Factory Managed Identities (recommended) or define your SQL Credentials (not recommended
  1. Create some annotations that allow you to monitor the workloads that use this linked service 
  1. Create some parameters if needed (covered in the following section). 
  1. Add custom code to the JSON definition 

Then, test the connection and finally, create the linked service. The new Azure Data Factory Linked Service becomes available on the list. You can see that it has 0 related items. 

Once you start creating Azure Data Factory datasets and pipelines, you can see related objects. 

Azure Data Factory Parameters for Linked Services 

Now, let’s say that you are working with dozens of databases or servers that are using the same connector. If you are not using parameters, you’ll end up with a really large number of linked services to maintain which becomes a daunting task. 

Parameters in Azure Data Factory Linked Services are available to minimize the effort you put in when working on and maintaining your solutions.  

Use parameters in conjunction with Azure Key Vault Secrets. 

Let’s create a parameter our previous connection. 

Click the connection and add a new parameter. 

Add a new parameter

Next, add a parameter. You can add a default value.  

Then, go to the secret configuration and add dynamic content. 

Add dynamic content

Click the previously created Azure Data Factory Linked Service Parameter and then finish. 

Click the linked service

 Now, the secret has been updated with the parameter. 

Updated secret with the parameter

If you click Test Connection, you will need to pass a parameter. 

Finally, you can save the Azure Data Factory linked service. 

When using datasets, you can define a different parameter value for your Azure Data Factory Linked Services which gives you a lot of flexibility. 

Define a parameter value

Summary   

To sum up, today you’ve learned how to create, use and add parameters to an Azure Data Factory Linked Service.  

What’s Next?       

In upcoming blog posts, we’ll continue to explore Azure Data Service features.           

Please  follow Tech Talk Corner on Twitter for blog updates, virtual presentations, and more!                 

comment [ 0 ]
share
No tags 0

No Comments Yet.

Do you want to leave a comment?

Your email address will not be published. Required fields are marked *