In vSphere 5 the VMFS filesystem has been updated to version 5 (currently 5.54). In vSphere 4.1 update 1 the VMFS version was 3.46.
In earlier versions of ESX, live upgrades of VMFS, or in-place upgrades, haven't been an option so to upgrade VMFS, basically a new LUNs had to be created and then VMs could be migrated to these new LUNs.
With vSphere 5, VMFS can be upgraded nondisruptively. This is done for each LUN by going to:
Datastore and Datastore Clusters -> Configuration -> Upgrade to VMFS-5.
It is a prerequisite that all connected hosts are running vSphere 5. The upgrade itself takes less than a minute (at least in a small test environment).
In VMFS 5, there is only one block size which is 1 MB. However, when upgrading from v3 to v5, the block size remains what it was before (see the last screendump). In the example below, the 8 MB block size is retained.
The new maximum LUN size is 64 TB - but a single .vmdk file can still not exceed 2 TB minus 512 bytes. The only way to have larger .vmdk's than 2 TB is to create an RDM and mount it as a physical device (as opposed to virtual). See this VMware whitepaper for further info.