Saturday, June 25, 2022

Cloning an Enterprise Scale git repo fails with error file name too long

 When working with Microsoft Enterprise Scale for Azure, it creates a rather long folder structure in the git repo. So usually when setting up a new developer machine for the first time and try to clone the repo, Git throws and error stating that file names are too long.

To fix this, open a PowerShell window as administrator (this can be from within VS code as well) and run the following command:

> git config --system core.longpaths true

And then you can rerun the git clone command.

Friday, June 24, 2022

Azure: Creating Private DNS Zones using copy loops with arrays and objects

 At current client, we potentially have to create +50 Private DNS Zones, so I was looking at optimizing the ARM templates as to not have hundreds of lines of code with basically the same info.

Furthermore the DNS zones have to be linked to two VNets (where the custom DNS servers reside).

This can be achieved using copy loop with a mix of arrays and objects.

The ARM template including parameters file is available on Github.

The list of private DNS zones (see full list here) is specified in an array and to loop through this list, a copy loop is used on the privateDnsZones resource, I have marked in red below the relevant lines.

For the virtual network links, a similar copy loop is used. In addition, an object is used as a parameter to make a bit nicer. The object contains two sets of info with link name and VNet resource ID for the two VNets, see more info on using objects here.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "privateZoneNames": {
            "type": "array"
        },
        "vnetSettings": {
            "type": "object",
            "metadata": {
                "description": "Object containing vlink name and vnet ID"
            }
        }
    },
    "resources": [
        {
            "apiVersion": "2018-09-01",
            "type": "Microsoft.Network/privateDnsZones",
            "name": "[concat(parameters('privateZoneNames')[copyIndex()])]",
            "location": "global",
            "dependsOn": [],
            "tags": {},
            "properties": {},
            "copy": {
            "name": "PrivateZoneCopy",
            "count": "[length(parameters('privateZoneNames'))]"  
            }
        },
        {
            "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
            "apiVersion": "2018-09-01",
            "name": "[concat(parameters('privateZoneNames')[copyIndex()], '/', parameters('vnetSettings').vlink[0].name)]",
            "location": "global",
            "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', concat(parameters('privateZoneNames')[copyIndex()]))]"
            ],
            "properties": {
                "registrationEnabled": false,
                "virtualNetwork": {
                    "id": "[parameters('vnetSettings').vlink[0].vnetId]"
                }
            },
            "copy": {
            "name": "vlinkWeuCopy",
            "count": "[length(parameters('privateZoneNames'))]"  
            }
        },
        {
            "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
            "apiVersion": "2018-09-01",
            "name": "[concat(parameters('privateZoneNames')[copyIndex()], '/', parameters('vnetSettings').vlink[1].name)]",
            "location": "global",
            "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', concat(parameters('privateZoneNames')[copyIndex()]))]"
            ],
            "properties": {
                "registrationEnabled": false,
                "virtualNetwork": {
                    "id": "[parameters('vnetSettings').vlink[1].vnetId]"
                }
            },
            "copy": {
            "name": "vlinkSwcCopy",
            "count": "[length(parameters('privateZoneNames'))]"  
            }
        }
    ]
}

--------------------------------
Param file:
--------------------------------

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "privateZoneNames": {
            "value": [
                "privatelink.blob.core.windows.net",
                "privatelink.vaultcore.azure.net"
            ]
        },
        "vnetSettings": {
            "value": {
                "vlink": [
                    {
                        "name": "vlink-vnet-conn-weu-001",
                        "vnetId": "/subscriptions/XXX/resourceGroups/rg-netw-conn-001/providers/Microsoft.Network/virtualNetworks/vnet-conn-weu-001"
                    },
                    {
                        "name": "vlink-vnet-conn-weu-002",
                        "vnetId": "/subscriptions/XXX/resourceGroups/rg-netw-conn-001/providers/Microsoft.Network/virtualNetworks/vnet-conn-weu-002"
                    }
                ]
            }
        }
    }
}

Thursday, June 23, 2022

Azure: Adding routes to route tables using Service Tags fail with CIDR error

At current client we were deploying some API Management service resources yesterday. For this we needed a direct route to the internet for management and monitoring services for the setup to function properly. This can be achieved by using Service Tags. The two service tags used are: ApiManagement and AzureMonitor.

It worked fine to add the routes to the route table using the Azure Portal. However, when trying to push the same via ARM templates I got an error from Azure DevOps stating the following (also see screenshot below):

"AzureMonitor of resource /subscriptions/XXX/resourceGroups/XXX/providers/Microsoft.Network/routeTables/udr-XXX-001/routes/AzureMonitor is not formatted correctly. It should follow CIDR notation, for example 10.0.0.0/24."

It turns out that the API version of the route table resource in the ARM template was using a too old apiVersion. When updating the apiVersion to the latest one, it works (not for template itself but specifically for the resource, see below (initially a 2017-08-01 version was used):





Wednesday, May 25, 2022

Azure VNet with nested VNet peering - ARM template

When creating a VNet peering between two VNets, this has to be configured separately on both VNets as it's two separate peerings. If instead you want to configure both VNet peerings in the same ARM template, this can be done using nested templates.

The resource type to use for peering is: "Microsoft.Network/virtualNetworks/virtualNetworkPeerings"

It's a standalone resource but is typically added together with VNet json.

The ARM template first adds the VNet peering on the current or the spoke VNet and the nested template creates an additional peering which is configured on the remote or hub VNet pointing back to the current/spoke VNet.

I've uploaded the full files to Github. This file (VNet-with-nested-peering.json and its parameters file) will deploy a VNet, a subnet, an NSG and the two VNet peerings. You just have to update the parameters file.

Below is shown the code specific to the VNet peerings (it doesn't format well below, but you can copy and paste it into a separate file if needed or preferably use the Github link above):

-----------------

{
            "name": "[concat(parameters('VNet_name'), variables('spokePeeringName'))]",
            "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings",
            "apiVersion": "2019-11-01",
            "dependsOn": [
                "[parameters('VNet_name')]"
            ],
            "properties": {
                "allowVirtualNetworkAccess": true,
                "allowForwardedTraffic": true,
                "allowGatewayTransit": false,
                "useRemoteGateways": false,
                "remoteVirtualNetwork": {
                "id": "[resourceid(parameters('HubSubscriptionId'), parameters('hubVnetRG'), 'Microsoft.Network/virtualNetworks', parameters('hubVnetName'))]"
                }
            }
        },
        {
            "apiVersion": "2017-05-10",
            "name": "nestedTemplate",
            "type": "Microsoft.Resources/deployments",
            "resourceGroup": "[parameters('hubVnetRG')]",
            "subscriptionId": "[parameters('HubSubscriptionId')]",
            "dependsOn": [
                "[resourceId(variables('currentSub'), variables('currentRg'), 'Microsoft.Network/virtualNetworks', parameters('VNet_name')) ]",
                "[concat(resourceId(variables('currentSub'), variables('currentRG'), 'Microsoft.Network/virtualNetworks', parameters('VNet_name')), '/virtualNetworkPeerings',variables('spokePeeringName'))]"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
                "contentVersion": "1.0.0.0",
                "resources": [
                    {
                    "name": "[concat(parameters('hubVnetName'), variables('hubPeeringName'))]",
                    "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings",
                    "apiVersion": "2019-11-01",
                    "properties": {
                        "allowVirtualNetworkAccess": true,
                        "allowForwardedTraffic": true,
                        "allowGatewayTransit": false,
                        "useRemoteGateways": false,
                        "remoteVirtualNetwork": {
                        "id": "[resourceId(variables('currentSub'), variables('currentRg'), 'Microsoft.Network/virtualNetworks', parameters('VNet_name')) ]"
                        }
                    }
                    }
                ]
                }
            }
        }

Monday, November 29, 2021

Create Azure ExpressRoute Circuit - ARM template

 There are multiple steps and design decisions that go into the deployment of an ExpressRoute. This article will cover (mostly) one of these steps which is the deployment of the ExpressRoute (ER) circuit.

An ExpressRoute circuit is a logical connection between your on-premises datacenters and Azure via a connectivity provider, see diagram below:


If you are designing an ER setup with multiple ER providers for DR, one ER circuit is required for each of these providers.

Once you deploy the ER circuit there will be a monthly running cost (both for metered and unmetered connections). 

To deploy the ER circuit, you can use an ARM template available on Github, see this link (this is one that I've used at a client) or use this version from MS Github. You can also deploy it manually from the portal if needed.

Note that to ensure that you enter the correct parameters in the parameter json, you can go through the steps of creating a new ER circuit via the Azure portal but instead of clicking "Create" on the last tab, click on "Download the template for automation". This will give you the ARM template including the parameters file. From here, you can copy the correct parameter information. Below are shown the main parameter options:


For port type there is a choice between Provider and Direct. Provider is the option described in this article (and which most companies would use) whereas ER Direct circumvents the service provider and connects directly to Microsoft equipment at the local peering edge location, see more here (however, a connectivity provider such as Telia or Interxion will still be used to do the actual work of connecting the physical equipment). With ER Direct you can get speeds up to 100 Gpbs, via a connectivity provider max bandwidth is 10 Gbps.

Peering location is the physical datacenter location of the connectivity provider. This location will typically be as close to the physical location for the client datacenters or main office location. You can see a list of providers and locations here. An example could be a Danish based company using Azure resources in West Europe region. A choice of local peering location could be Copenhagen via the provider Interxion. When traffic reaches Copenhagen on provider managed lines it will continue on the Microsoft backbone to Amsterdam (West Europe).

For the SKU, there is choice between Standard and Premium. You can see the differences here. Likely, Standard SKU will suffice for most clients.

For Billing Model there is a choice between Metered and Unmetered, see prices here. For Metered connection, there is a cost for egress traffic (traffic leaving Azure) but not ingress.

When the connectivity provider has established the last mile and connected the client datacenter to a local peering edge, they will ask for a Service Key to be able to configure the circuit on their side. The Service Key is available from the portal once the ER circuit is provisioned, see screenshot below:


After creating the ER circuit, Private Peering (as opposed to Microsoft peering for Office 365) must be configured. See details here (This step can be set up either by the connectivity provider or or you so ask the provider how they usually do it).

Then verify Private Peering.

After this, create a connection to the ER GW (similar to a VPN gateway but for ExpressRoute) from portal: Circuit -> Connections -> Add. This can also be done via ARM, see this link.


Tuesday, October 5, 2021

Azure Firewall - Forced Tunneling not working for internet bound traffic

 At current client we have a use case that is not widely used. It's built around the Microsoft Enterprise-Scale AdventureWorks Hub/Spoke network infrastructure. The traffic from spokes is filtered via Azure Firewall to a VPN gateway in the Hub and from there to on-premise datacenters. Azure Firewall Forced tunneling is configured so that all traffic to on-prem including internet bound traffic is routed this way and will ultimately exit via an on-premise web proxy. The usual configuration would be to route internet bound traffic directly to the internet from the Azure Firewall (AZ FW).

Since we're currently building the environment we had only tested the connectivity to on-premise local addresses and not internet bound traffic. When testing we could see that traffic from the spokes reached the AZ FW and was correctly allowed by the network rules. But from there it sort of disappeared.

Then we set up a VPN packet capture on two of the VPN gateways (VPN GW), one was a control where traffic was not filtered via the AZ FW. We could see that internet bound traffic never reached the VPN GW and so the problem was introduced before that point either in the FW itself or the UDRs.

Microsoft Support suggested that we remove the UDR/route table associated with the AzureFirewallSubnet (which contains a default route, 0.0.0.0/0, to the virtual network gateway) since this should already be pushed from the VPN GWs.

Removing the UDR actually fixed the issue (so that no route tables are associated with the AzureFirewallSubnet). So having a double configuration which in itself is not incorrect results in internet traffic being dropped by the AZ FW.

While testing this we couldn't see the local source IP addresses of the test VMs in the on-prem firewall. Instead we saw an interface/IP address from the AzureFirewallSubnet range (which was not the configured IP of the local AZ FW interface but from same subnet so it belongs to the FW). The reason for this is that SNAT is configured by default for internet bound addresses but not for local addresses. This can be changed under AZ FW Policies and set to 'Never'. With this, the original source IPs are visible in the on-prem FW (they are not SNAT'ed or masked). See the configuration of SNAT in screen shot below.



 


Friday, October 9, 2020

Join a VM to an active directory domain in Azure with PowerShell

 At current client we have a hybrid setup with an on-prem datacenter and VPN connections to Azure. The on-prem active directory has been extended to Azure with an additional two domain controllers (classic active directory).

We have a generic virtual machine deployment template in PowerShell and want to be able to have VMs automatically join the domain as part of the deployment process.

A first learning is to make sure that the IP addresses of the DC's are specified as custom DNS servers on the VNets where they reside (assuming they also function as DNS servers). Otherwise, the VMs won't be able to resolve the domain (VNet -> DNS servers -> Custom).

Up front we have verified that a VM can be manually joined to the domain from the VM itself.

The PowerShell command for setting the domain is: Set-AzVMADDomainExtension

The MS documentation is not very thoroughly described for this command, unfortunately.

We got pretty far using this guide instead.

When using that example and trying jo join, the PowerShell script kept hanging and timing out. From the VM itself in the event log, it just stated NetJoin action failed. But no helpful error description.

We had to adjust two things to make it work:

1. For the JoinOption parameter, instead of using 0x00000001, we used 0x00000003 (which is similar to setting 0x00000001 and 0x00000002). This specifies joining a domain instead only a workgroup (option 1) and option 2 creates an account on the domain.

2. By using the Invoke-AzVMRunCommand command we set the DNS suffix from inside the VM using PowerShell before running the domain join command.

This works in our setup. The code examples can be seen below (first script calls the second script). It can be added to an existing VM deployment script or be run as a standalone script post VM install.

Join domain

# Update variables
$VMName = "XXVMname"
$ResourceGroupName = "XX-RGname"

# Do not update these variables
$DomainName = "XXdomainname.local"
$ServiceAcct = "XXdomainname\XXserviceaccountname"
$secretnameDomain = "XXnameofsecretinKeyVault"
$JoinOpt = "0x00000003" # specifies the options to join a domain, 0x00000001 + 0x00000002
$OU = "OU=XX,OU=XX,DC=XX,DC=Local" # Update string with corrrect OU path (not a requirement)
$kvname = "XXnameofKeyVault"
$LocationName = "westeurope"
$ScriptPathDNSsuffix = ".\SetDnsSuffixInVM.ps1"

Write-Host "Set DNS suffix in VM. $(Get-Date)"
# Set DNS suffix in VM (for domain join to work)
Invoke-AzVMRunCommand -ResourceGroupName $ResourceGroupName `
 -VMName $VMName `
 -CommandId 'RunPowerShellScript' `
 -ScriptPath $ScriptPathDNSsuffix

# Get service account password from key vault
$ServiceAcctPassword = (Get-AzKeyVaultSecret -vaultName $kvname -name $secretnameDomain).SecretValueText | ConvertTo-SecureString -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ($ServiceAcct, $ServiceAcctPassword);

Write-Host "Joining domain $DomainName. $(Get-Date)"
Set-AzVMADDomainExtension -DomainName $DomainName `
 -VMName $VMName `
 -ResourceGroupName $ResourceGroupName `
 -Location $LocationName `
 -Credential $Credential `
 -OUPath $OU `
 -JoinOption $JoinOpt `
 -Restart `
 -Verbose
Write-Host "Script done at $(Get-Date)"

Set DNS Suffix in VM (SetDnsSuffixInVM.ps1)

$DnsSuffix = "XXDomainname.local"

$interfacealias = get-dnsclient | Where-Object {$_.InterfaceAlias -eq "Ethernet"}
set-dnsclient -Interfaceindex $interfacealias.Interfaceindex `
-ConnectionSpecificSuffix $DnsSuffix `
-RegisterThisConnectionsAddress $True