Updating ExpressRoute Circuit fails with error: Peering/Circuit was recently updated by service provider

 Recently I had to make some minor updates to an ExpressRoute Circuit via Azure DevOps. The change was to apply a read-only resource lock via code so no actual changes were introduced to the circuit configuration itself. Still, when running the pipeline it fails showing the error below:

"Error: The Peering/Circuit was recently updated by the service provider and is not currently in sync" (see screenshot below).

Looking in the portal after this failed push, the ExpressRoute (ER) circuit was in failed state and the error message suggests to refresh the circuit, see screenshot below. When clicking refresh, the status goes back into green. Note that even if the portal shows an error, the circuit itself did not drop any traffic (which is good as this is a pretty critical component).

This particular ER circuit has a private peering which has been configured by the service provider (this can either be done by the customer or by the service provider). However, we knew that the service provider had not recently updated the circuit so it did not make much sense.

We raised a ticket to Microsoft and they suggested to specify the "gatewayManagerEtag" in the code, see more info here. I didn't find much info around this parameter other than it's a string and supposedly it can be used to ensure that the service provider or the customer don't accidentally override a common setting with an older value. To identify the current value I went to the portal and exported the ARM template for the ER circuit (the value was 7).

After specifying this parameter, the push went through with no problems. And at the same time, we specified the private peering configuration info in the ARM template, which was not there from the beginning (as it had been set by the service provider). Additionally, we tried updating another ER circuit where we ourself had initially configured the private peering in the ARM template and where the gatewayManagerEtag value was empty or just showing as "". This also works fine and we included this on all circuits for consistency.

I've uploaded an example ARM template to GitHub which includes private peering configuration, the gatewayManagerEtag and a read-only resource lock, see links below (remember to update the content of the parameters file):

er-circuit-w-private-peering.json

er-circuit-w-private-peering.parameters.json

Create Azure ExpressRoute Circuit - ARM template

 There are multiple steps and design decisions that go into the deployment of an ExpressRoute. This article will cover (mostly) one of these steps which is the deployment of the ExpressRoute (ER) circuit.

An ExpressRoute circuit is a logical connection between your on-premises datacenters and Azure via a connectivity provider, see diagram below:

If you are designing an ER setup with multiple ER providers for DR, one ER circuit is required for each of these providers.

Once you deploy the ER circuit there will be a monthly running cost (both for metered and unmetered connections). 

To deploy the ER circuit, you can use an ARM template available on Github, see this link (this is one that I've used at a client) or use this version from MS Github. You can also deploy it manually from the portal if needed.

Note that to ensure that you enter the correct parameters in the parameter json, you can go through the steps of creating a new ER circuit via the Azure portal but instead of clicking "Create" on the last tab, click on "Download the template for automation". This will give you the ARM template including the parameters file. From here, you can copy the correct parameter information. Below are shown the main parameter options:

For port type there is a choice between Provider and Direct. Provider is the option described in this article (and which most companies would use) whereas ER Direct circumvents the service provider and connects directly to Microsoft equipment at the local peering edge location, see more here (however, a connectivity provider such as Telia or Interxion will still be used to do the actual work of connecting the physical equipment). With ER Direct you can get speeds up to 100 Gpbs, via a connectivity provider max bandwidth is 10 Gbps.

Peering location is the physical datacenter location of the connectivity provider. This location will typically be as close to the physical location for the client datacenters or main office location. You can see a list of providers and locations here. An example could be a Danish based company using Azure resources in West Europe region. A choice of local peering location could be Copenhagen via the provider Interxion. When traffic reaches Copenhagen on provider managed lines it will continue on the Microsoft backbone to Amsterdam (West Europe).

For the SKU, there is choice between Standard and Premium. You can see the differences here. Likely, Standard SKU will suffice for most clients.

For Billing Model there is a choice between Metered and Unmetered, see prices here. For Metered connection, there is a cost for egress traffic (traffic leaving Azure) but not ingress.

When the connectivity provider has established the last mile and connected the client datacenter to a local peering edge, they will ask for a Service Key to be able to configure the circuit on their side. The Service Key is available from the portal once the ER circuit is provisioned, see screenshot below:

After creating the ER circuit, Private Peering (as opposed to Microsoft peering for Office 365) must be configured. See details here (This step can be set up either by the connectivity provider or or you so ask the provider how they usually do it).

Then verify Private Peering.

After this, create a connection to the ER GW (similar to a VPN gateway but for ExpressRoute) from portal: Circuit -> Connections -> Add. This can also be done via ARM, see this link.