Azure: Bulk update secrets in Azure key vault using ARM

 At current client we have a requirement to periodically rotate some 50 secrets in Azure key vaults that are used for storing the shared keys for S2S VPN connections.

For keys (not secrets) there is an auto rotation option that is now GA, see link, however it doesn't apply to secrets.

There's an option to use logic apps to rotate secrets but the MS documentation I could find is not very good.

At first I was looking to write a PowerShell script that pulls the new secrets from a csv file and then applies them to a key vault, but it ended up being easier just creating an ARM template that does the same.

The secret names and secret values are stored in the parameters file (in the file there are a couple of test secrets for demonstration purposes). The name of the KV should also be updated in the parameters file. The ARM template itself does not have to be changed.

If you add a secret in the param file that already exists in the KV, it will overwrite the existing secret and add a new/current version and keep the previous version(s) as Older versions, see screenshot below. If you have secrets already in the KV that are not defined in the param file, these secrets will not be changed or deleted. And if you add new secrets in the param file that are not in the KV currently, they will be added.

The template is available on Github, the files are:

deploy_secrets.json

deploy_secrets.parameters.json

Powershell cmd to run ARM template.txt

Azure Firewall - Forced Tunneling not working for internet bound traffic

 At current client we have a use case that is not widely used. It's built around the Microsoft Enterprise-Scale AdventureWorks Hub/Spoke network infrastructure. The traffic from spokes is filtered via Azure Firewall to a VPN gateway in the Hub and from there to on-premise datacenters. Azure Firewall Forced tunneling is configured so that all traffic to on-prem including internet bound traffic is routed this way and will ultimately exit via an on-premise web proxy. The usual configuration would be to route internet bound traffic directly to the internet from the Azure Firewall (AZ FW).

Since we're currently building the environment we had only tested the connectivity to on-premise local addresses and not internet bound traffic. When testing we could see that traffic from the spokes reached the AZ FW and was correctly allowed by the network rules. But from there it sort of disappeared.

Then we set up a VPN packet capture on two of the VPN gateways (VPN GW), one was a control where traffic was not filtered via the AZ FW. We could see that internet bound traffic never reached the VPN GW and so the problem was introduced before that point either in the FW itself or the UDRs.

Microsoft Support suggested that we remove the UDR/route table associated with the AzureFirewallSubnet (which contains a default route, 0.0.0.0/0, to the virtual network gateway) since this should already be pushed from the VPN GWs.

Removing the UDR actually fixed the issue (so that no route tables are associated with the AzureFirewallSubnet). So having a double configuration which in itself is not incorrect results in internet traffic being dropped by the AZ FW.

While testing this we couldn't see the local source IP addresses of the test VMs in the on-prem firewall. Instead we saw an interface/IP address from the AzureFirewallSubnet range (which was not the configured IP of the local AZ FW interface but from same subnet so it belongs to the FW). The reason for this is that SNAT is configured by default for internet bound addresses but not for local addresses. This can be changed under AZ FW Policies and set to 'Never'. With this, the original source IPs are visible in the on-prem FW (they are not SNAT'ed or masked). See the configuration of SNAT in screen shot below.