Hands-on labs - that I did...

This post is mostly for myself to keep track of the labs that i did at VMworld 2010 and what struck me as useful features...

vSphere 4.1 - new features

Storage IO shares and limits. They're not quite there yet with Storage DRS but this is a first step.

DRS feature - possibility to keep one or more VMs bound to a specific host. This is practical in relation to licensing issues where you pay per physical core/socket in a cluster.

8 concurrent VMotions!

Better HA info with new Cluster info link. It will aggregate relevant HA info and display it in a window.

Scripting ESXi installation with PXE boot and get the files over the network from a tftp server looked pretty straight forward.

VMware View 4.5 - install and configure

Components: Main View Manager server (not the correct name, I believe..). Transfer server for offline clients. View agent installs on the VM. View client to connect to View Manager.

vCloud Director - install and configure

Basically a VM that install on top of your VI and communicates with vCenter. Works as administration unit and self service portal to customers/admins. Installs on a RHEL 5 U4 or up, 64-bit. Licensing is done per VM. vShield Manager is required.

Update Manager

Practical for host patching of course... Also relevant for upgrading VMware Tools and HW version. Tools and HW version can be done in a single script.

Thinapp 4.6 - new features

The new feature that IE6 can be packaged easily is very useful. If you package an app on WinXP it can be run on other OS'es, e.g. Win7, without modification. Every time I play around with Thinapp it always strikes me why we don't use it more...

PowerCLI 4.1

One of the first examples is a script to go through your VMs and find and delete all snapshots older than a given date - e.g. older than seven days. This example is spot on and something that we're working on imlementing as well.

vShield Zones

One thing that I noted in this lab is that vShield has a built in load balancer which was easy to configure. This could probably be a good substitute for MS NLB.

VMworld 2010 - Hands-on labs, first impressions

There's been a lot of talk about the hands-on labs at this years VMworld. Here in Copenhagen there are around 240 thin clients with each two screens running up against two datacenters in Florida and Virginia with a dedicated 100 Mbit line over the Atlantic. There's also a fail-over possibility to a datacenter in Europe, should an issue occur. The whole solution is based on a number of Lab Manager installations with a custom interface on top of it which has been built for the purpose.
So far, I have taken four labs, and I must admit that I'm impressed. It works so well that you're not even thinking about what is going on under the hood. What you're actually experiencing is that as soon as you're choosing a lab, then almost instantly you have two ESX servers and a number of VMs provisioned for you. A VMware employee told me that part of the custom code is calculating the most popular labs, and then pre-deploying a number of these up front as to reduce waiting time.
Furthermore, the whole user experience is pretty cool. They've made a GUI for choosing the labs, and on flatscreens around in the room you can see statistics such as most popular labs and total number of VMs created and labs completed (see pictures below).

Restart of ESX management agents

This is just a post to remember the commands for restarting the management agents on an VMware ESX server:

#service mgmt-vmware restart

#service vmware-vpxa restart (the HA agent)

Both of these agents can be restarted without affecting VM operation. Restarting them can be a useful step in troubleshooting if vCenter has trouble connecting to a host or if you experience HA errors.

Here's link to KB article

For restarting mgmt agents in ESXi, this can be done via the console menu interface, see link above.

Console-setup - service console tool for network config in ESX4

As of ESX 4.0 U2 a new tool for configuring network in the service console (COS) has been introduced. If you're not too comfortable with CLI then this might come in handy. The tool will give you a numbered menu and you can list and configure nics, vswitches, vswifs, etc.

Here's a link to a VMware KB article that presents the tool.

To run the tool, type console-setup in the COS.

Menu entry 1, 2, and 3 will show the output of esxcfg-vswif -l, esxcfg-nics -l, and esxcfg-vswitch -l respectively.

Menu entry 5 will let you configure your service console without having to remember any of the commands. Pretty neat..

Here's a link on how to do it the old school way.

What does ESX stand for?

Not too long ago I was doing some general VMware introduction to a number of colleauges from our Chinese branch. One of the guys asked what ESX was short for. And I had actually no idea. It's a product I've been working with for several years and yet it hadn't ocurred to that ESX was probably more than just a three letter name.

Today, I saw an article on acronyms on Yellow-Bricks which reveals the secret. Furthermore, there's a link to a video interview with Mike DiPetrillo which elaborates on the matter (about 20 minutes in...)

So:

ESX: Elastic Sky X

GSX: Ground Storm X

The two names was invented by some marketing people hired by VMware. VMware didn't like it too much so they shortened it to ES and GS and the X was added just to make it sound more technical!

Disaster recovery: Procedure in case of site failure

Here's a short example of a procedure for recovering a VMware cluster from a site failure. The example scenario consists of two ESX4 hosts on replicated storage divided on seperate locations. There's no automatic failover for storage between sites, manual breaking of the mirror is required.

1.	Log into vCenter and verify whether or not storage is available for the cluster. If storage is unavailable, create an incident ticket for the storage group with priority urgent and with a request to: “Manually break the mirror for the “"Customer X" replicated storage group” used by ESXA and ESXB” The ticket should be followed by a phone call to the storage day/night duty to notify of the situation.
2.	When mirror has been broken, rescan remaining hosts in the cluster. This rescan can possibly time out. If this happens, reboot the hosts. After the rescan/reboot all shared LUNs will be missing on the hosts. These should be added/mounted manually from the console (step 3) (in ESX4u1 there's a bug in the add "storage" wizzard, so it doesn't work from the vSphere client, see this post for more info)
3.	Putty to each of the hosts and run the following commands: #esxcfg-volume –l This will list available volumes. For each volume, run the following command: #esxcfg-volume –M label or UUID> For example: #esxcfg-volume –M PSAM_REPL_001 See screendump below for further exemplification:
4.	From the vSphere client, for each of the available hosts go to Configuration -> Storage and click “Refresh”. Verify that all LUNs appear as before the site failure
5.	Power on all VMs
6.	Done. In this situation, storage will run from the secondary site. The storage group will be able to reverse the replication seamlessly at a later stage when failed site is operational again. This does not require involvement from the VMware group.

Site redundancy with manual breaking of storage mirror

We have just installed a site redundant cluster for a customer. The cluster consists of two ESX4 hosts on replicated EMC CLariion storage. The ESX servers as well as the storage reside on different locations (preferrably we would have liked to have done it with storage virtualisation and seamless storage failover ala Datacore or SVC, but this was not an option..).

The site redundancy is enabled by using replicated storage. Should the site with the active LUNs fail, then the storage mirror can be broken manually and operation can be resumed on the remaining site.

One thing we discovered we that resignaturing of the LUNs is no longer necessary, as it was in previous versions when a mirror had been broken. This means that LUNs can be remounted directly without modifications, see Fibre Channel SAN Configuration guide pp. 74-76.

Earlier, you had to first break the mirror, then resignature your LUNs with the advanced feature LVM.resignature and then add the LUNs. This changed the UUID (and the label on the LUNs for that matter) which means that all VM had to be manually reregistered in virtualcenter. This is a bit time consuming and not something you want to spend yor time on in a disaster scenario.

In vCenter, you can use the "add storage" wizzard to remount the LUNs. However, there's a known bug in the software so it does not work. In stead, it has has to be done from command line with the following command (rescan the HBAs first. if it hangs, then reboot):

# esxcfg-volume -l (to list available volumes)

# esxcfg-volume -M (to persistently mount volume)

See this post for example site recovery procedure