OSCustomization for VM builds using PowerCLI

VM builds are probably the boring tasks that are repeatedly done by any vSphere Admins be it in Lab or Production, so it’s a good idea to automate them and spend time on better things. In this post i will be showing the script that i have been using since many years when i come across those VM builds and it only takes less than 5 min to create a VM – love automation!

In order to use the script, we need to actually create a OS customization first and this is a one time task and also we need to have a Windows VM Template created. Once you have it just bring up the PowerShell console and use the below script, the script prompts for VM name, Reserved IP, Windows Version and VM port group information and once you enter them and that’s it.

This script deploys a VM from template, joins the Domain;Uses VMware built-in OS Customization. Tested for Windows Server 2008 R2 and 2012 R2.
Script prompts for name of the VM you want to deploy, IP, VLAN and Windows Version
Use PowerCLI v6 and above preferably one pulled from PSGallery, PowerShell v4 and above

[string]$VMName = (Read-Host "Enter the Name of the VM "), 
[string]$VMIP = (Read-Host "Enter the IP reserved for the VM "),
[string]$PortGroup = (Read-Host "Enter the VLAN for the VM, example 10 "),
[string]$Winver = (Read-Host "Enter the Windows version, example 2012 ")

#Import PowerCLI Module
Import-Module VMware.PowerCLI

# Connect to vSphere02-SC9.virginam.com
Connect-VIServer vcenter.vmmaster.local

# Choose the right nicmapping for the vlan
if ($PortGroup -eq 10)
$vmvlan = 10
$pgnicmapping = "vlan10" 
$vmGateway = "<IP Here>"
elseif ($PortGroup -eq 20)
$vmvlan = 20
$pgnicmapping = "vlan20"
$vmGateway = "<IP Here>"

# Choose the right template
if ($Winver -eq 2008)
$vmTemplate =Get-Template -Name Template_Win_2008R2
$vmTemplate = Get-Template -Name Template_Win_2012R2

#Select Prod Cluster
$vmCluster = Get-Cluster -Name <Cluster Name>

#Select a Random Datastore connected to Prod Cluster
$vmDatastore = $vmCluster | Get-Datastore | Where {($_.ExtensionData.Summary.MultipleHostAccess -eq "True") -and ($_.FreeSpaceMB -gt "102400") -and ($_.State -ne "Maintenance")} | Get-Random

#Clone the Customization Spec
$vmSpecification = Get-OSCustomizationSpec -Name Windows_Builds | New-OSCustomizationSpec -Name WindowsTemp -Type NonPersistent

#Update the portgroup and gateway based on the VLAN entered
$tempnicmapping = Get-OSCustomizationSpec $vmSpecification | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -ipmode UseStaticIP -IpAddress $VMIP -SubnetMask -DefaultGateway $vmGateway -Dns <DNS IPs here>

#Build the VM
New-VM -Name $VMName -Template $vmTemplate -OSCustomizationSpec $vmSpecification -ResourcePool $vmCluster -Datastore $vmDatastore -DiskStorageFormat Thin

Write-Host " New VM Details `n Name:`t $VMName `n OS:`t Windows Server $Winver R2 `n IPAddress:`t $VMIP `n Datastore:`t $vmDatastore" -ForegroundColor Gray

Start-Sleep -Seconds 30

#Set the right portgroup for the VM
Get-VM $VMName | Get-NetworkAdapter -Name "Network adapter 1" | Set-NetworkAdapter -NetworkName $pgnicmapping -Confirm:$false -StartConnected:$True

#Power On the VM for the customization to start
Write-Host " Powering on the VM $VMName" -ForegroundColor Green
$vm = Start-VM $VMName -Confirm:$false

Write-Host "`n Waiting for the customization to complete! Monitor the console to see the progress.......... `n" -ForegroundColor Green

while (Get-VM $VMName | where {$_.PowerState -eq "PoweredOn"})
$vmEvents = Get-VIEvent -Entity $VMName
$succeeded = $vmEvents | Where { $_.GetType().Name -eq "CustomizationSucceeded" }
$failed = $vmEvents | Where { $_.GetType().Name -eq "CustomizationFailed" }

if ($failed)
Write-Host " Customization failed!" -ForegroundColor Red
return $False

Write-Host " Customization Complete!" -ForegroundColor Green
return $True

Start-Sleep -Seconds 10 

If you are interested in procedure for creating Customization Specification, here it is. Adding batch files that can install and configure config management tools like Puppet, Chef in the Run Once section will actually ensure all the needed configurations are present for the application.

This slideshow requires JavaScript.

Hope this was informative. Thanks!


How to find the Duplicate VM UUID’s

I recently came across an issue with the VMware SRM where there was a VM having the same UUID in both the primary and recovery site. This VM was a clone from the primary site and was never powered on the recovery site but was rather cloned to template and was used for new builds, over time, everyone forgot about the cloned VM and we recently started seeing issues with the recovery plan. We were unable to test and cleanup the recovery plan. We started seeing the below PANIC message in the SRM logs.

Panic: VERIFY d:/build/ob/bora-3037005/srm/src/recovery/settings/vmRecoverySettingsRepository.cpp:1291

This is a known issue in SRM 6.1 and KB related to this can be found here. The KB does not provide a way to look for the duplicate UUID’s so here is what we used to find the VM UUIDs on both Primary and Recovery vCenter servers.

Run the below code to get the VM’s and their UUIDs in a csv, once you have the csv, its very easy to identify the duplicates. Once you have the VM identified, Shutdown the VM, browse to the VM datastore and backup the vmx file of the VM and edit the value for uuid.bios and re upload to the datastore and power on the VM. Just to make sure it does conflict with the other VMs again, verify the UUID you used is not present in the csv file.

Here are step by step instructions on editing the vmx files Link1 Link2

$vmlist =get-vm | sort name 

forEach ($vm in $vmList) {
       $vmName=$vm | %{(Get-View $_.Id).Name}
       $vmUuid=$vm | %{(Get-View $_.Id).config.UUID}
       echo "$vmName,$vmUuid" | Out-File -Append <path to csv>

Here is the sample output from my lab


Hope this was informative and useful. Thanks!

Install VCSA 6.7

With the release of vSphere 6.7, like everyone, i was also interested in trying out the installs in my lab and figure out what was changed and how things really looked. To start with, i first installed the VCSA 6.7 in my lab and below is how it went.

This is my current VCSA 6.5 lab build.


Initially i wanted to a upgrade of the existing appliance but i realized i did a ova install of VCSA 6.5 in my workstation. The upgrade only works for actual install of VCSA on an ESXi host. So, i dropped the idea of upgrading and just decided to do a new install of VCSA 6.7 in the current lab and at some point i want to migrate my vSAN lab to VCSA6.7 and just shutdown the VCSA 6.5. So here we go!


From the ISO, navigate to the install folder and run the setup file and the below screen pops up. This is similar to the previous versions with some attractive icons. Choose install here.


Continue with the next few screens, select the type of deployment below and hit Next.



Since i am already installing my VCSA in a vCenter environment, i will be selecting my existing vcenter server for the appliance install.


Now, name the VM for VCSA and  set a root password.


Select the size of the deployment and hit Next


Select the Datastore for the VM, i have a VSAN datastore in my existing vCenter, so i will make use of that


Assign IP address and DNS Server details for the VM and hit Next


Review the settings, sit back and relax after hitting Finish.


OVA deploy should be in progress…..


OVA deploy is complete, to configure it click Continue on the below screen. In a browser, navigate to the appliance management at port 5480.


Click setup on the below screen.


Click Next


The IP and VM settings should automatically pop up in here from Stage 1, verify and click next.


Configure the SSO Domain


Hit Finish and wait for the Stage 2 to complete.


Once complete, login back to the appliance management page to view the vCenter build and other details.


Thats all for today! I am working on a blog series for new features in vSphere 6.7 so i can catch up on the new features and also help others.

Hope this was informative. Thanks!

Part 8 – Docker Commands with VIC

Now that we have a VCH deployed into the environement, it’s time we actually run some native docker commands and see if the containers are being created.

Observe that the Docker Engine here is linux based.


Browse through the vsanDatastore to see the files related to VCH that we deployed and an images folder is created, this is where all the images are pulled to when docker pull is run.


Now let’s pull an image from docker hub. When running docker native commands, we should always mention the host with -H as below. Now run couple of containers from the pulled busybox image.

docker -H --tls run busybox
docker -H --tls run --name vmmaster busybox


To avoid specifying the values each time, we can actually set the VIC parameters as environment variables.


After having pulled the busybox image and punching in docker run, we see the containers running in the HTML client as below.


I did another ubuntu image pull. All the native commands are still good here. Below are a few of them


As discussed earlier, containers appear as running VMs in the virtual infrastructure. The container poweron and poweroff must not be done from VMware client. It is always managed from docker client using native commands.


Hope this was informative and helped you learn a bit about VMware Integrated Containers. Thanks!

Part 1 – vSphere Integrated Containers

Part 2 – VIC Lab Setup

Part 3 – Deploy VIC Appliance

Part 4 – Obtain vCenter Certificate Thumbprint

Part 5 – Install Client Plugins on VCSA for VIC

Part 6 – Opening Ports on ESXi Hosts

Part 7 – Deploy VCH to vCenter ServerPart 8 – Docker Commands with VIC

Part 8 – Docker Commands with VIC

Part 7 – Deploy VCH to vCenter Server

VCH is the key component of the VIC as this is the resource pool where all the container VMs and the VCH Endpoint VM resides here. VCH is deployed using the vic-machine utility and this can be run from any Windows, Mac or Linux machines. The binaries needs to be downloaded to the machine that runs these commands. Below are the prerequisites for deploying VCH.

  • Download binaries from the VIC appliance
  • Enable DRS on the cluster that is going to host VCH
  • All the hosts have a common datastore to enable easy migration between hosts; we are using vSanDatastore here.
  • Create a vDS with port group named ‘vic-bridge’. Ensure this port group is only used for one VCH and a new port group would be needed if a new VCH has to be deployed. This is for the isolation of the container VMs
  • All the ESXi hosts are licensed with Enterprise Plus
  • DHCP enabled in the network or assign a network profile to the port group with range of IPs.


From the machine where binaries are run the below command

vic-machine-windows.exe create --target vcsa.vmmaster.vic --user "administrator@vsphere.vic" --password VMware1! --bridge-network vic-bridge --image-store vsanDatastore --no-tlsverify --thumbprint CC:E1:DA:C8:93:4F:60:F9:13:EC:38:38:10:DF:54:CD:FC:61:44:AE


A successfull deploy of VCH displays the message with details on how to connect to the docker API on the VCH. Observe a new resource pool is created and also there is a VCH Endpoint VM for each VCH deployed.



The same can be viewed in the HTML 5 client


In case the VCH deploy did not succed use the below to delete the partially deployed resource pools and any other configurations.

vic-machine-windows.exe delete --target administrator@vsphere.vic:VMware1!@vcsa.vmmaster.vic --thumbprint CC:E1:DA:C8:93:4F:60:F9:13:EC:38:38:10:DF:54:CD:FC:61:44:AE --name virtual-container-host --force

To verify the VCH installation run below from any docker client in the network

docker -H --tls info


Hope this was informative. Thanks!

Part 1 – vSphere Integrated Containers

Part 2 – VIC Lab Setup

Part 3 – Deploy VIC Appliance

Part 4 – Obtain vCenter Certificate Thumbprint

Part 5 – Install Client Plugins on VCSA for VIC

Part 6 – Opening Ports on ESXi Hosts

Part 7 – Deploy VCH to vCenter ServerPart 8 – Docker Commands with VIC

Part 8 – Docker Commands with VIC

Part 6 – Opening Ports on ESXi Hosts

Port 2377 is used for the communication between VCH and ESXi hosts. The name of the firewall rule is vSPC, if at all the rule is disabled for some reason, one must configure the firewall using other methods like web client and CLI.

Opening 2377 for outgoing connections in ESXi opens 2377 for inbound connections on VCH. Download the VCH binaries from the management portal. The binaries has the vic-machine utility needed for the VCH installation.


unzip the files to view the binaries and vic-machine utility.


Navigate to the download location of the binaries and run the below command from the elevated command prompt and make sure to use the right vic-machine file based on the OS you use.

vic-machine-windows.exe update firewall --target vcsa.vmmaster.vic --user "administrator@vsphere.vic" --password VMware1! -compute-resource My_Cluster --thumbprint CC:E1:DA:C8:93:4F:60:F9:13:EC:38:38:10:DF:54:CD:FC:61:44:AE --allow

Since we are going to VCH to a cluster managed by vCenter, we will execute the firewall open command against the vCenter server. This opens the port on all the ESXi hosts in the cluster.


Hope this is informative. Thanks!

Part 1 – vSphere Integrated Containers

Part 2 – VIC Lab Setup

Part 3 – Deploy VIC Appliance

Part 4 – Obtain vCenter Certificate Thumbprint

Part 5 – Install Client Plugins on VCSA for VIC

Part 6 – Opening Ports on ESXi Hosts

Part 7 – Deploy VCH to vCenter ServerPart 8 – Docker Commands with VIC

Part 8 – Docker Commands with VIC

Part 5 – Install Client Plugins on VCSA for VIC

The next step in the installation is to Install the client plugin for VIC. VIC plugin is very much integrated with the HTML 5 client than the web client. Enable shell on VCSA.

If you have a environment variable set as part of previous installation of VCH, the installation will fail. Ensure it is deleted.

Open a browser and use the VIC appliance IP with port 9443 to view the exact file name. Make a note of the vic tar file including the version. In my case it is vic_1.2.1.tar.gz.

vic20Enable shell and SSH on the VCSA and open a putty session as root adn run the below commands. The first command downloads the tar file and the second one unzips and sets permissions. Navigate the vic/ui/VCSA directory.

curl -kL -o vic_1.2.1.tar.gz
tar -zxf vic_1.2.1.tar.gz


Run the install.sh script to start the plugin installation. Enter the details as and when prompted. Have the vCenter thumbprint handy.


After the installation is complete, restart the web client and HTML 5 services.


Refresh the webclient and HTML client to view the vSphere Integrated Containers plugin.



Hope this was informative. Thanks!

Part 1 – vSphere Integrated Containers

Part 2 – VIC Lab Setup

Part 3 – Deploy VIC Appliance

Part 4 – Obtain vCenter Certificate Thumbprint

Part 5 – Install Client Plugins on VCSA for VIC

Part 6 – Opening Ports on ESXi Hosts

Part 7 – Deploy VCH to vCenter ServerPart 8 – Docker Commands with VIC

Part 8 – Docker Commands with VIC