Azure: Adding routes to route tables using Service Tags fail with CIDR error

At current client we were deploying some API Management service resources yesterday. For this we needed a direct route to the internet for management and monitoring services for the setup to function properly. This can be achieved by using Service Tags. The two service tags used are: ApiManagement and AzureMonitor.

It worked fine to add the routes to the route table using the Azure Portal. However, when trying to push the same via ARM templates I got an error from Azure DevOps stating the following (also see screenshot below):

"AzureMonitor of resource /subscriptions/XXX/resourceGroups/XXX/providers/Microsoft.Network/routeTables/udr-XXX-001/routes/AzureMonitor is not formatted correctly. It should follow CIDR notation, for example 10.0.0.0/24."

It turns out that the API version of the route table resource in the ARM template was using a too old apiVersion. When updating the apiVersion to the latest one, it works (not for template itself but specifically for the resource, see below (initially a 2017-08-01 version was used):

Activate new Visual Studio subscription and start using credits in a new Azure AD tenant

 If you want to test code in Azure or you just want a sandbox to try things out, then having a Visual Studio (VS) subscription (formerly MSDN) is a good idea. This gives you a certain amount of dollars (or credits) to spend each month in Azure.

There are two types of subscriptions:

• Visual Studio Professional (50 USD/month in credits)
• Visual Studio Enterprise (150 USD/month in credits)

The subscription is personal to the employee and is typically acquired by the employer and assigned to the employee on the employee email address (you will receive an email with activation info from Microsoft). See pricing details here.

Activate Visual Studio subscription

Once you have been assigned a VS subscription, it has to be activated. Go to below link to activate:

https://my.visualstudio.com/

Log in with your company email, if you don't have an account already, then create one (using company email).

Activate the monthly credits under the Featured Benefits section, see below:

You can view your subscription details under the Subscriptions tab:

Create new Azure AD tenant

When you log into Azure using your company credentials, you will typically see the company's AD tenant under Azure Active Directory, e.g. companyname.onmicrosoft.com. Since you'll likely have limited permissions in this tenant (AD tenant is similar to an AD domain, just in the cloud. It's also referred to as Directory) and since the aim it to create an isolated sandbox for testing, then a new AD tenant has to be created and the VS subscription associated with this tenant.

To create a new AD tenant, go to:

http://portal.azure.com

Go to: Create a resource -> Identity -> Azure Active Directory (see below):

Give your new AD tenant (or organisation) a name and click create:

Note that this will not affect the existing company AD tenant. Even though you create it while logged in with your company email, it will have no cost for the company and will only be manageable by you. I note this as I have discussed this with both colleagues and clients where they worry that they might break something in the company Azure AD. They won't.

Move your VS subscription to the new AD tenant

Once your new Azure AD is created, you can go to your VS subscription and change the directory so that the subscription will be associated with your new tenant.

• Go to Subscriptions and choose your VS subscription
• Under "Overview" choose "Change directory"
• Choose your newly create Azure AD tenant (as opposed to the company AD tenant)
• Click OK, see below two screen dumps:

Note that it can take a few hours before the changes kick in.

To switch around between tenants and also to choose your default tenant, click on your profile in upper right corner and choose your directory/tenant and/or default tenant, see below:

That's it! Now you can start using your monthly credits in a tenant that you have full control over.

To get an overview of current spending and to know when the monthly credits reset, see this post.

Azure Firewall drops traffic to on-prem S2S VPN

We're currently setting up an Azure Firewall at a client site. Initial implementation was done by following the MS documentation: 

https://docs.microsoft.com/en-us/azure/firewall/deploy-ps

After deployment and after attaching a given subnet to the default route table (to force all outbound traffic to pass through the Azure Firewall), then there was no communication from the Azure subnet and to the local VPN gateway (S2S VPN to on-premises). Other subnets that weren't attached to the route table worked fine.

It seemed that new default route (see example below) might be overriding the existing default routes in Azure. And that the way forward would be to create additional routes to specify traffic to the VPN.

$routeTableDG = New-AzRouteTable `

-Name Firewall-rt-table ` -ResourceGroupName Test-FW-RG ` -location "East US" ` -DisableBgpRoutePropagation #Create a route Add-AzRouteConfig ` -Name "DG-Route" ` -RouteTable $routeTableDG ` -AddressPrefix 0.0.0.0/0 ` -NextHopType "VirtualAppliance" ` -NextHopIpAddress $AzfwPrivateIP ` | Set-AzRouteTable

It turns out that the standard routes were fine. Only the default route to the internet is overridden (which is expected), the other two remain in use:

These are the standard routes that Azure creates:

"Each virtual network subnet has a built-in, system routing table. The system routing table has the following three groups of routes:

Local VNet routes: Directly to the destination VMs in the same virtual network.

On-premises routes: To the Azure VPN gateway.

Default route: Directly to the Internet. Packets destined to the private IP addresses not covered by the previous two routes are dropped."

The problem was that the property setting: "Propagate gateway routes" was set to "No", see below. This means that the VPN gateway routes are not visible/propagated to the subnets. To turn it on you can either do it from the portal or configure it via Powershell.

To do this in Powershell, simply remove the line: -DisableBgpRoutePropagation from the New-AzRouteTable command so it looks like below :

-Name Firewall-rt-table ` -ResourceGroupName Test-FW-RG ` -location "East US" #-DisableBgpRoutePropagation

This will set the property to: "False" which in the portal corresponds to "Yes".  You can see the "false" setting under Route table -> Export template.

When done, the published routes from the virtual gateway become visible under Route tables -> "your route table" -> Effective routes, like below, and traffic will flow to the on-prem site via VPN as well.

How to check Visual Studio limits and reset date in Azure

With a Visual Studio subscription (formerly MSDN) in Azure you get a monthly usage quoata which can be used for testing purposes. There are two types of subscription:

• Visual Studio Professional (50 USD/month)
• Visual Studio Enterprise (150 USD/month)

The subscription is personal to the employee and is typically acquired by the employer and assigned to the employee on the employee email address.

You can manage your subscription and activate it from here:

https://my.visualstudio.com/

Once you start using the subscription (typically assicioated with a new Azure AD tenant that you create), you can see the current usage in https://portal.azure.com/ by going to:

Subscriptions -> Overview -> Scroll to the bottom of the page

Below image shows limit in local currency.

If you want to know when your monthly reset is (if you're running low), then on the same page click on the "Manage" button marked in green above.

This will take to https://account.azure.com/ where you can see reset date and cost breakdown, see below:

Installing AzCopy v10 on Linux

AzCopy v10 is a command line tool (CLI) for copying files between clients and Azure Blob.

In previous versions the application had to be installed on the client machine. With v10 this is no longer required. AzCopy is now a single file that is downloaded and can be run standalone.

Go here to download the correct version for your OS.

To install:

Go to the above link and copy the download link for Linux OS (will be used with wget command below):

Log in to your server as a regular user

Download the tar.gz file to your home folder:

$ wget https://aka.ms/downloadazcopy-v10-linux

Untar the archived files:

$ tar -xvf downloadazcopy-v10-linux

Go into to the untar'ed folder to locate azcopy executable:

$ cd azcopy_linux_amd64_10.0.4/

Verify that you can run the program:

$ ./azcopy

This will show you which version you're running.

If you have already installed a previous version of azcopy, this will be called unless you specify the full path to the new version.

You can remove the old version simply by deleting the executable (I'm fairly sure that this is sufficient):

$ sudo rm /usr/bin/azcopy

Next copy the new version in to the same location (/usr/bin is likely already in your PATH it can be run from everywhere):

$ sudo cp azcopy /usr/bin/

Go to the root folder, run azcopy and verify that it's v10:

$ cd /

$ azcopy

Alternatively, you can leave the old version and rename the new one to azcopy10. Or you can create an alias for the file and put in your ~/.bashrc_aliases file, see link here for info.

See this link for example copy commands.

Installing Blobfuse in Ubuntu to mount Azure Blob as file share

Azure Blob is around a factor x3 cheaper than Azure Files, it doesn't have the current 5 TB limitation that Azure Files has (100 TB is in tech preview), and it has tiering (hot/cold/archive) to reduce storage prices even further. The problem is, though, that Blob is object storage and not block storage so if you need to mount Blob as a file system from e.g. a Linux server you have to put something in front of it.

Avere and Blobfuse lets you do this. With Blofuse it's important to note that it's not 100% POSIX compliant, see link here for limitations. This means that linux permissions won't apply (chmod, chown) and symbolic links will not work either. We will be given full read/right permissions to the container (as the storage account key is used for authentication).

As a side note, you can do copying of data from CLI with Azcopy v10 and Rclone but they work with their own set of commands and do not remote mount Blob as a folder or share.

Prerequisites:

Note that to set this up you need an SSD disk or a dedicated portion of memory (as a cache) in the VM where you set this up. This example will show how to use memory as a ram cache.

Also, you need to have a storage account created in MS Azure (that we will mount).

This MS guide has been used to set up Blobfuse. Ubuntu 18.04 has been used in this setup but the guide shows how to use other OS'es as well.

Install/configure:

Ssh or log in to your server

Su to root:

$ sudo su -

Configure the Microsoft package repository:

# wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb

(or if you're on Ubuntu 16.04: https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb)

# dpkg -i packages-microsoft-prod.deb
# apt-get update

To install Blobfuse:

# apt-get install blobfuse

Mount an 8 GB temp ram drive for cache (the VM used for this has 16 GB so I'm using half for the cache):

# mkdir /mnt/ramdisk

# mount -t tmpfs -o size=8g tmpfs /mnt/ramdisk

# mkdir /mnt/ramdisk/blobfusetmp

# chown jakob /mnt/ramdisk/blobfusetmp

Go to your home folder:

# cd /home/jakob

Create below file (it doesn't have to be in your home folder but it can be):

# touch fuse_connection.cfg

Edit the file with nano:

The below

# nano fuse_connection.cfg

Add the following content to the file (this info can be found in Azure Portal -> Storage Accounts -> click your storage account -> Access keys. The container name is the name of the virtual folder that you're mounting. This container should already have been created either on in the Azure Portal or the Azure Storage Explorer):

accountName jakobsstorageaccount

accountKey wbMTSWXXXXXXXXXXXXXXXXXXusRtAA==

containerName jakobscontainer1

Save the file and exit.

Change permissions so only root can access it:

# chmod 700 fuse_connection.cfg

Make a folder where the content of container will be mounted:  

# mkdir /mnt/blobfuse/jakobscontainer1

Mount the container:

# blobfuse mnt/blobfuse/jakobscontainer1 --tmp-path=/mnt/ramdisk/blobfusetmp --config-file=/home/jakob/fuse_connection.cfg -o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120

That's it. Now test that you can list the content of the Blob container:

# ls -alh /mnt/blobfuse/jakobscontainer1

From here you can use cp or Rsync to copy files back and forth.

Free linux cloud VM with Amazon Web Services (AWS)

Recently, Amazon announced that you can get a free linux VM for one year in their public cloud solution - Amazon Web Services (AWS). They call it a Micro Instance and it's got something like 1 vCPU, 600 MB of memory, and 10 GB storage, see specs here. You get full access to the VM via SSH but there's no console access as such.

So I decided to give it a try.

First, you need to create an AWS account (there's a link on the front page..). They need a valid creditcard for that. Then you log into the AWS Management Console. This requires you to register again. They had implemented a rather odd security feature where they call your mobile phone and you have to punch in a pin-number to confirm. I must admit that, for testing purposes, this wasn't the most smooth registration process.

Once into the AWS Management Console you're presented with a number of tabs. The first one is Amazon S3 which is an online file placeholder (i guess like an FTP server). To create your VM, go to the Amazon EC2 tab and click on Launch Instance (see below). This process is fairly simple. It is not quite easy, though, to see exactly which one is the free edition, but I just chose the minimum specs available to be on the safe side. And look for something like linux and Micro Instance.

Firewall rules are easy to configure via the web interface. You can add some pre-defined ports such as mail, web, etc. Port 22 is enabled by default.

A KPI keyset is generated (for authentication purposes) and you can download the .pem file to your local harddrive. They give an example of howto login via ssh from a console and use the generated key. Example:

ssh -i keyname.pem root@vmname.eu-west-1.compute.amazonaws.com

If you use this command will receive a login error as root cannot login directly. So just change 'root' in front of the @ with the, in the error message, suggested 'ec2-user'.

Once logged in you can execute commands as root with the 'sudo' command. It will not prompt for a password. Or alternatively use sudo -i to get a root console. But you can't su - root.

If you want to use Putty to acces the VM directly, then you have to convert the .pem file to a .ppk file. This is easily done using this guide.

To use the .ppk file, open Putty and go to SSH -> Auth and browse to the directory where you stored the file. And then you connect to the VM (saving the profile will save you some time at next login..). There's no password.

The same .ppk file can also be used for WinSCP which is handy for uploading files directly to the VM.

As you have a public DNS name, this can be used to create an eiasier to remember C-name DNS that you can point to the generated machine name.

So far so good. Now there's access via SSH. Then I tried to configure a simple web server. I'll describe that in the next post.