[spacer] [spacer] [spacer]

Sign up with your email address to be the first to know about new publications

[mc4wp_form id="4946"] [spacer]

Azure Data Factory Alerts

Do you know how to enable Azure Data Factory alerts in case of failure? This is one of the great capabilities to consider when developing solutions. 

In this blog post, you will learn how to: 

  • Create and configure alerts for Azure Data Factory 
  • Monitor alerts 
  • Opt-out an action group

You can create the alerts by using Azure monitor alerts. You can access them from your Azure Data Factory or Azure Alerts.

Alerts

New Alert Rule 

First, click “New alert rule” to get started. 

New alert rule

When creating alerts, you need to configure four different options: 

  • Scope – the service(s) that you are monitoring. 
  • Condition – the rule that needs to be met to trigger the alert. 
  • Action group – the action to execute after the alert is triggered. 
  • Alert rule details – the name and description of the alert.

Scope 

If you have entered the alerts option from your Azure Data Factory, the information will be filled in automatically. 

Scope

However, if you have entered the alerts from Azure Monitor Alerts, you have to select the services that you want to monitor. 

Alerts from Azure Monitor Alerts

Condition 

When configuring the condition, all the metrics available for Azure Data Factory are going to be displayed. The number of metrics has increased significantly during the past few years, as well as analytics for information in the log. 

Configure signal logic

In this case, to enable the alerts in case of failure, just filter by failed and select “Failed pipeline runs metrics.” This will cover the majority of failures as data flows and SSIS packages run within pipelines. 

Search by failed alerts

When selecting the metric, you need to configure how to aggregate the information and the frequency of evaluation. Then, click done. 

In the following picture, I have selected the alert to evaluate failures every 15 minutes (for the previous 15 minutes.) 

Additionally, you can filter by specific pipeline names of error type if required. This is really useful when you require more control over the criticality of your data movement. 

Filtering by pipeline name 

Filter by pipeline name

Filtering by error type 

Filter by error type

Finally, after saving the condition, you will see that it is available in the configuration page. 

Action Group 

Action groups are independent objects from alerts and you can reuse them. 

An action group can be defined by using the following option: 

Or in this case, directly from the alert. 

For the action group, you must configure a few options. 

  • Basics – basic details of the action group 
  • Notifications – delivery notification method (email, SMS, call, etc.) 
  • Actions – an additional action to trigger (for example Azure functions or Webhooks) 
  • Tags 
  • Review and create 

Basics 

Let’s define some basic information for the action group. 

Define basic information for the action group

Notifications 

In this case, I am the only member of this action group and will configure the email and SMS delivery options. Each delivery method has different costs. 

Actions 

For this tutorial, I won’t define any actions, but the option is available to you. 

Tags, review and create 

Once you are ready, create the action group. 

Once you are back on the alerts configuration page, you will see the action group. It’s also possible to define multiple action groups. 

Alert rule details 

Finally, the last step is to define a name, a description, and the severity level for your alert.  

There are different severity levels defined by Microsoft in Azure: 

  • 0 = Critical 
  • 1 = Error 
  • 2 = Warning 
  • 3 = Informational 
  • 4 = Verbose 
Alert rule details

Testing the alert 

To test the alerts, I have a simple pipeline with a lookup activity that tries to divide by zero. 

After triggering the alert, I received the following notifications: 

Email 

SMS 

Monitoring the alerts 

In case you want to know which alerts have been triggered, you can do this by going to the alerts option. 

Monitoring the alerts

Did someone accidentally add you to an action group? 

When you’re added to an action group, you will receive a notification. You can disable it if you don’t want to receive any alerts. 

Email 

SMS 

An option for disabling the alerts is available in the alert configuration. 

Summary 

In this blog post, we reviewed how to easily enable alerts for Azure Data Factory failures using the different metrics available within Azure Monitor. 

Final Thoughts 

Complementing your Azure Data Factory solutions with good monitoring capabilities is a practice that you need to take into account for new or mature environments. 

Having alerts helps you respond to critical and non-critical issues in production environments. 

What’s Next? 

In upcoming blog posts, we’ll continue to explore some of the features within Azure Services.   

As always, please leave any comments or questions below. 

If you haven’t already, you can follow me on Twitter for blog updates, virtual presentations, and more! 

Check out these other blog posts:

No Comments Yet.

Do you want to leave a comment?

Your email address will not be published. Required fields are marked *