How to use dd to work with VMDKs stored on VMFS

Table of content :

Todo:
– add mapping examples to compare the results of vmkfstools -c with the dd equivalents.
– add the section about extracting partitions


Why use dd at all ?

Sometimes the webinterface options give unexpected results – like changing an eagerzeroed thick VMDK to lazy zeroed.
Sometimes vmkfstools just does not want to cooperate …
In those cases you may consider to use dd – as dd will work always.
But dd is a really dangerous command.
If you get your values incorrectly you can truncate your VMDKs – so dont try any of this commands when you are in a haste or under the influence of drugs.

Do not use any of the commands I describe here against VMDKs stored on vSAN or NFS – this howto is for regular VMFS only.
If you create VMDKs with vmkfstools or the GUI you will get 2 files:
name.vmdk and
name-flat.vmdk
Obviously dd only creates the flat.vmdks so you have to create the associated descriptorfile yourself. See the first chapter …

Safety first …
dd commands can truncate your flat.vmdks – so always add the parameter conv=notrunc to your commands unless you know for sure that this parameter MUST be skipped.
In all other cases this parameter will not hurt.
You will notice that I always use a blocksize of one MB for the commands I use.
This is a reasonable choice as ESXi also uses this blocksize when ever it creates, modifies or reads VMDKs.


… back


How to create a vmdk-desciptorfile for an existing flat.vmdk ?

Please do NOT follow the instructions in the VMware Knowledgebase – see https://kb.vmware.com/s/article/1002511
That procedure creates a brand new flat-VMDK just to delete it afterwards again.
Thats a bad idea – especially in a recovery scenario.
Instead connect to the ESXi with WinSCP.
Go to the directory where you stored the flat.vmdk.
Right click the flat.vmdk and look up the size in bytes.
Divide that value by 512 to get the <size in sectors>
Divide <size in sectors> by 16065 and round down the result and note this as <cylinders>
Also lookup the extact name of the flat.vmdk <name-flat.vmdk>
Now you should have 3 variables:
<name-flat.vmdk>
<size in sectors>
<cylinders>

This is all you need to know to create a new descriptor.vmdk from scratch:
Create a new file with the embedded editor of WinSCP and paste this  text into name.vmdk.
———————————————————————————————————————

# Disk DescriptorFile
version=1
encoding=”UTF-8″
CID=12345678
parentCID=12345678
createType=”vmfs”

# Extent description
RW <size in sectors> VMFS “<name-flat.vmdk>

# The Disk Data Base
#DDB

ddb.adapterType = “lsilogic”
ddb.geometry.cylinders = “<cylinders>
ddb.geometry.heads = “255”
ddb.geometry.sectors = “63”

———————————————————————————————————————
Note:
All parameters starting with ddb. are basically optional.
The calculation for the geometry is valid for all SCSI-VMDKs larger than 1 GB.
The geometry for IDE-VMDKs always use this values:
ddb.geometry.cylinders = “16383”
ddb.geometry.heads = “16”
ddb.geometry.sectors = “63”
Dont worry about the ddb.adapterType too much – the parameter is deprecated and I have not seen problems when you get this wrong.
If unsure – just skip it or use the value “lsilogic”
I always use the CID-value 12345678 for newly created descriptorfiles as it is the easiest way to make sure you really get a number with 8 digits.

NOTE: a descriptor file created like this is good for the following cases:
– you created a VMDK with dd
– you accidentaly deleted a descriptor-file for a VMDK without snapshots
– you need a new descriptor in a recovery scenario
A descriptor made like this will not be sufficient when the VMDK has snapshots – in this case you have to lookup the CID-values.


… back


How to create an eager zeroed thick VMDK ?

Create a 8GB vmdk with vmkfstools:
vmkfstools -c 8192m 8gb-eager.vmdk  -d eagerzeroedthick

Create a 8GB vmdk with dd:
dd if=/dev/zero of=dd-8gb-eager-flat.vmdk bs=1M count=8192 conv=notrunc

This command will take quite a while as the complete size of the new flat.vmdk will be wiped with zeros.


… back


How to create a lazy zeroed thick VMDK ?

Create a 8GB vmdk with vmkfstools:
vmkfstools -c 8192m 8gb-lazy.vmdk  -d zeroedthick

Create a 8GB vmdk with dd:
sorry – I have no idea at the moment.
I also see no reason to use lazy zeroed thick VMDKs at all – so my motivation for further research is limited.



… back


How to create a thin VMDK ?

Create a 8GB vmdk with vmkfstools:
vmkfstools -c 8192m 8gb-thin.vmdk  -d thin

Create a 8GB vmdk with dd:
dd if=/dev/zero of=dd-8gb-thin-flat.vmdk bs=1M seek=8191 conv=notrunc

This will just add one MB of zeros at the very end of the VMDK.
The space in between the start of the flat.vmdk and the one MB of zeros at the end will be thin provisioned.


… back


How to expand a VMDK with an eager zeroed fragment ?

1. figure out the current size:
– using cli: ls -l name-flat.vmdk
– divide the size in bytes by 1048576 to get the <current size in MB>
– calculate the value for count: <new size in MB><current size in MB> = <X>

2. then run the command
dd if=/dev/zero of=name-flat.vmdk bs=1M seek=<current size in MB> count=<X> conv=notrunc
This will add X MBs at the end of the existing flat.vmdk and will wipe the added space with zeros.
The space in between the former end of the flat.vmdk and the new end will be eager zeroed provisioned.


… back


How to expand a VMDK with a lazy zeroed fragment ?

Sorry – I dont know how to do this at the moment.
In my humble opinion this makes no sense at all so I do not see any reason to research this problem.


… back


How to expand a VMDK with a thin fragment ?

1. figure out the current size:
– using cli: ls -l name-flat.vmdk
– divide the size in bytes by 1048576 to get the <current size in MB>
–  calculate the seek-value by this formula: <new size in MB> – 1 = X

2. then run the command
dd if=/dev/zero of=name-flat.vmdk bs=1M seek=<X> count=1 conv=notrunc
This will just add one MB of zeros at the very end of the VMDK – The space in between the former end of the flat.vmdk and the new end will be thin provisioned.
This option is probably much safer and more failsafe as the eager zeroed expand – as you only need <new size in MB> – 1 and so run no risk to overwrite content of the original vmdk.


… back


How to cut an existing VMDK ?

WARNING – DANGEROUS IF YOU MISCALCULATE THE NUMBERS – YOU HAVE BEEN WARNED

Using this procedure only makes sense to revert an accidental expand of a VMDK using GUI – functions or vmkfstools -X and it only makes sense if:
– you are 100 % sure that the original MBR or GPT has not been adjusted to the new size
– you are 100 % sure that the original partitions have not been expanded
– you still know the original size

If you do no longer have the original descriptor vmdk  to lookup the old size from the RW line:
RW <size in sectors> VMFS “<name-of-flat.vmdk>
then try to find an old vmware.log to look up the value.
Search for DISKLIB-VMFS …
If you found the value for <size in sectors> divide it by 2048 to get the old <old size in MB>
Subtract 1 from <old size in MB> and note the new value as <old size in MB -1>
Next extract the last MB of the original size by
dd if=<name-of-flat.vmdk> of=scissor.bin bs=1M skip=<old size in MB -1 > count=1
Inspect the newly created file scissor.bin and check if this really looked like the last MB of the original size.
hexdump -C scissor.bin | less
If you are satisfied use scissor.bin and reinject it at its original location so that it cuts off the flat file after it – this is done by using dd without the conv=notrunc parameter.
To do that use
dd of=<name-of-flat.vmdk> if=scissor.bin bs=1M seek=<old size in MB -1 > count=1
Please note that if and of parameters are used in reverse now.
Also note that the skip parameter to extract scissor.bin now is used for the seek parameter.
Do not try a short cut and use if=/dev/zero for the cut-command.
Using a 1MB sized scissor.bin is definetely the safest approach.

A final warning: if you are unsure about the values – do not use this approach.

… back


How to extract a partition from an existing VMDKK ?

coming soon …
This is useful to split VMDKs with several partitions in new “one partition per virtual disk” – VMDKs which are easier to handle.

… back