Category Archives: VMware

Power Up Your HomeLab With PowerCLI

Introduction

As a follow on to the shut down home-lab script, I have also written a power up script for when power is restored to your home-lab and you want to get it up and running again quickly.  This script will assume your hosts are powered on and waiting, but nothing else has happened yet.

 

The Script

The script can be downloaded below, make sure you read though it fully and understand how it works.  I am not responsible for anything going wrong.!

At the top of the script, you will need to change the settings to match your own home-lab.

[string]$vCenter_Server                   = "svr-vc"
[string]$vCenter_Server_UserName          = "administrator"
[string]$vCenter_Server_Password          = "********"

[string]$Known_ESX_Host                   = "xxx.xxx.xxx.xxx"
[string]$Known_ESX_Host_UserName          = "root"
[string]$Known_ESX_Host_Password          = "********"

[string[]]$Critical_Servers_Startup_Order = ("live-dc","svr-dc","svr-sql", "svr-vc")

Lines

  1. The name of your virtual centre server
  2. The username you use to connect to your virtual centre server
  3. The password for the above username
  4. .
  5. The IP address of one of your ESXi hosts. This is explained below
  6. The username of the root account of for the above host
  7. The password for the root account
  8. .
  9. The servers in your critical folder, in the order they should be powered up

 

Starting from the known host (the same as in the shut down script), it will power up your critical servers in the order specified on line 9.  Once they are fully powered on (it checks that the VMware tools are available) it will check and wait for the VMware VirtualCenter Server service to be up and running.  Once running, the script will then start all the remaining virtual machines.

 

Download

Download the script from the link below, and again, make sure you test it before putting it live.!

download-fileStartUpEntireHomeLab.ps1

Shutdown Your HomeLab With PowerCLI

Introduction

If you run a large home-lab like I do, shutting it down in an emergency can be a slow process, especially if you are panicking about your UPS staying up long enough.  You  DO have a UPS don’t you.!?

I had written a PowerCLI script for myself that was custom to my specific lab.  I have since made it more generic so that everyone can use it if they wanted to.  The script does rely on one specific criteria for your home-lab though.

 

Home-Lab Layout

vc-foldersWithin the vCenter VMs and Templates view, you can organise your VMs into folders to help separate them into logical groups.  This has no bearing on the VMs themselves, its purely for your information.

As you can see by this image, I have a folder called Critical that I use to hold my important VMs.  These are…

  • live-dc : The DHCP and DNS server for my home network,
  • svr-dc : The domain controller, as well as DHCP and DNS for my private lab network,
  • svr-sql : The Microsoft SQL server for my private network.  This holds all the databases for my lab,
  • svr-vc : The VMware virtual centre server for my home-lab.

This Critical folder is important in my script, as it controls which VMs are not shut down immediately, but must wait until the end and be shut down in a specific order.

 

The Script

The script can be downloaded below, make sure you read though it fully and understand how it works.  I am not responsible for anything going wrong.!

At the top of the script, you will need to change the settings to match your own home-lab.

[string]$vCenter_Server                    = "svr-vc"
[string]$vCenter_Server_UserName           = "administrator"
[string]$vCenter_Server_Password           = "********"

[string]$Known_ESX_Host                    = "esx1"
[string]$Known_ESX_Host_UserName           = "root"
[string]$Known_ESX_Host_Password           = "********"

[string]$Critical_Folder                   = "Critical"
[string[]]$Critical_Servers_Shutdown_Order = ("svr-vc", "svr-sql", "svr-dc", "live-dc")

[int]$Shutdown_TimeOut = 180 # Seconds

Lines:

  1. The name of your virtual centre server
  2. The username you use to connect to your virtual centre server
  3. The password for the above username
  4. .
  5. The name of one of your ESXi hosts.  This is explained below
  6. The username of the root account of for the above host
  7. The password for the root account
  8. .
  9. The critical folder mentioned above (not case sensitive)
  10. The servers in your critical folder, in the order they should be shut down in
  11. .
  12. How long to wait for a clean shut down before killing power to a virtual machine

 

Specific ESXi Host Name

The reason for a specific host name in the script, is so that when you come to power on your home-lab, you will know exactly which host will hold your critical VMs.  The script will migrate all powered off VMs as well as your critical ones to this host before powering everything down.  If you have a number of hosts it’s helpful if you don’t need to check each one for your domain controller is virtual centre server.

 

Download

Download the script from the link below, and again, make sure you test it before putting it live.!

ShutdownEntireHomeLab.ps1download-file

More Pushover Configuration

Introduction

After successfully configuring Pushover with PRTG, I moved on to configuring my Synology DiskStation (easy) and VMware vCenter (harder) installation to also notify me via Pushover.

 

Register An Application

In both cases, I registered a new application within the Pushover console.  I did this so that each alert would come though with their own specific icon and description.  See my previous blog posting on how to do this.

I used the following icons for each application :

  • synology Synology
  • vmware VMware

 

Synology DiskStation Notifications

syno-notifications-1This is was the easiest change to make, simply…

  1. Login to your DiskStation,
  2. Open the Control Panel,
  3. Select the Notification option,
  4. Choose either (or both) of the following tabs…
    • Email
    • Push Service
  5. Enter your Pushover API email address
    • [user-key]+a=[api-key]@api.pushover.net

 

VMware vCenter Alerts

Setting up these alerts were a lot harder.  It took a lot of trial and error.  Hopefully this will save you the same hassle.

Before you start, make sure you have the Microsoft dotNet framework version 4 or later installed, as well as PowerShell 4 or later.  Next, we’ll create a folder and two files on your vCenter server…

  1. On the C: drive, create a folder called Scripts,
  2. Create two plain text files and enter the following code…

send-alert.bat

"c:\windows\system32\windowspowershell\v1.0\powershell.exe" c:\scripts\send-alert.ps1 -title '%1' -message '%2'

send-alert.ps1

param(
    [string]$title,
    [string]$message
)

$parameters = @{
    token    = "[api-key]"
    user     = "[user-key]"
    priority = "0"
    title    = $title
    message  = $message
}

$parameters | Invoke-RestMethod -Uri "https://api.pushover.net/1/messages.json" -Method Post

Remember to enter your own api-key and user-key into the .ps1 script file.

 

vmware-definitionsvCenter Configuration

Next we need to configure VMware vCenter to run the batch file whenever an alert condition is met.  At last count there are 68 different alert definitions within vCenter.  You could change each on manually, but that would take all day.  If you have VMware PowerCLI installed (and you really should have) you can script this.

Manual Steps…

  1. vmware-alarm-4Open VMware vSphere Client, and login to your vCenter server,
  2. Select the very top level entry in the tree view on the left,
  3. Choose the Alarms tab along the top,
  4. Click the Definitions button,
  5. The list of 68+ definitions are list.  Double-click on one of them.  I’ll choose Datastore usage on disk,
  6. Select the Actions tab,
  7. Click Add,
  8. From the new entry that appears,
    • change “Send a notification email
    • to “Run a command
  9. Enter the following code into the Configuration column…
c:\scripts\send-alert.bat "{targetName}  >  {eventDescription}" "{triggeringSummary}"
  1. Click OK
  2. Rinse and repeat for the remaining definitions, or…

Automatic Script…

  1. Open a PowerCLI command window, and connect to your vCenter server
  2. Copy and paste the following code, and let it run…
$al = Get-AlarmDefinition
ForEach ($a in $al) {
    Get-AlarmDefinition -Name $a | New-AlarmAction -Script -ScriptPath 'c:\scripts\send-alert.bat "{targetName}  >  {eventDescription}" "{triggeringSummary}"'
}

Make sure you get the single quotes ( ‘ ) and double-quotes ( ” ) correct

For a list of variables you can use, check the VMware documentation.

 

Results

Once you have alerts working, you should get alerts through looking something like this…

space-alert

VMware Mobile Monitoring

Introduction

If you want to quickly check on your virtual infrastructure and you’re not next to your PC/Laptop, what do you do.?

Well, VMware have released a tool that will allow you to monitor you hosts and guests.

 

watchlistVMware Mobile Watchlist

(Description taken directly from Google Play Store)

vSphere Mobile Watchlist enables secure vSphere infrastructure monitoring and remediation directly from your smartphone. With Watchlist, VMware administrators will be able to log in to a vCenter Server or ESXi host directly and choose virtual machines and hosts from inventory to create targeted views of objects and their properties. Remediate directly from the device with power and management operations, and delegation of tasks to onsite colleagues with linked relevant Knowledge Base (KB) articles.vSphere Mobile Watchlist brings the following key capabilities to vSphere administrators on the go:

  • EASILY CREATE WATCHLISTS:
    Search for and select a subset of VMs and hosts from your VMware vCenter Server and/or ESXi inventory to monitor in one or multiple watchlists.
  • VMs and HOSTS AT A GLANCE:
    Review the status of selected Watchlist VMs and hosts from your device including state, configuration details, resource usage, health alerts, view of the VM console, and related objects.
  • DISCOVER:
    Alerts are linked to pertinent diagnostic information from the VMware Knowledge Base, as well as articles from the Web.
  • REMEDIATE REMOTELY:
    Remediate problems directly from the device by powering on/off, resuming/suspending, connecting/disconnecting, or restarting VMs and/or hosts — or for situations where on-site remediation is required, share alert(s) along with recommended solutions in an email to team members back at the datacenter.

vSphere Mobile Watchlist is compatible with Android 4+.

 

Screen Shots

Below are some screen shots of my setup, showing my three hosts a a few of my running VMs…

Login Screen   Main Watchlist Screen (List View)   Main Watchlist Screen (Grid View)   ESX1 - Overview   ESX1 - Command Options   ESX1 - Task List

 

Download

You can download and install it from the Google Play Store using the link below…

store-google

 

 

NTP Time Service

Introduction

Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems…  NTP is intended to synchronize all participating computers to within a few milliseconds of Coordinated Universal Time (UTC).

Taken from Wikipedia

Making sure you have a reliable and accurate time source can help with mitigating issues on your network.

 

Problem

One of my ESXi hosts was replaced recently, and using my automatic build script I had the new replacement up and running quite quickly.  What my script doesn’t do however, is configure the NTP settings with the host and because of this, it’s internal clock was about 6 hours ahead of the rest of the network.

My PRTG Network Monitoring system is a VM that, via DRS, can move freely amongst my three hosts depending on the current workload.  At 5:00 am the other morning, my monitoring server was moved to the offending host.  This screwed up its monitoring results.

As the VM was migrated to the new host, the time suddenly jumped forward by about 6 hours.  The OS then took over sometime after and jumped back to the correct time.  This time flip-flop caused large gaps in the monitoring results, and left me scratching my head for a while figuring out why.!

 

How to fix the above issues

There were two issues at play here, the host NTP settings were not correct, and the guest VM was set to synchronise it’s time with the host.

 

Configure Host NTP Settings

To enable configure each hosts time setting…

  1. Select your host (if you have more than one, you will need to do this on each one in turn),
  2. Select the Configuration tab along the top of the main section,
  3. Down the left hand menu, near the middle, select Time Configuration,
  4. Click Properties at the top right,
  5. From the first window that appears, make sure the NTP Client Enabled option is selected,
  6. Click Options,
  7. From the General item, choose the middle option, choose Start and stop with host,
  8. Select the NTP Settings item, click Add, and enter the IP address or host name of your chosen NTP host,
  9. Tick the option Restart NTP service to apply changes.

NTP-1

NTP-2   NTP-3

If you don’t have an internal NTP time source, the best option is to use 0.pool.ntp.org1.pool.ntp.org2.pool.ntp.org, and 3.pool.ntp.org.  See the link below for more information, or for geographically local sources…

http://support.ntp.org/bin/view/Servers/NTPPoolServers

 

Disable ‘synchronise guest time with host’

I had my home-lab domain configured to use a specific time source, and to push that out to all the Windows machines.  This is a good idea to configure.  What I had done wrong, was allow the virtual machines guests to have their clocks synchronised with the hosts.  This is generally a bad idea.

  1. Right-click a virtual machine, choose Edit Settings…,
  2. Choose the Options tab along the top,
  3. Select the VMware Tools settings,
  4. Un-tick the Synchronize guest time with host option.

While you are there, it’s also a good idea to tick the other option : Check and upgrade Tools during power cycling.

 

If you have a lot of hosts to change, this is going to be a very tedious task.  Thankfully, there is a script that will handle this for you.

The script below will set both options for you:

  • Line 12: Check and upgrade Tools during power cycling,
  • Line 13: Synchronize guest time with host.

Save the script as a .PS1 file, and execute it within PowerCLI.  Remember to change the first line to enter your vCenter server name or IP address.

Connect-VIServer "ENTER VCENTER SERVER HERE"

# Query for the VM guests
$VMGuests = Get-VM

# Loop through your VM guests, set the VM Tools upgrade checkbox and the Sync Time checkbox to true
ForEach ($VMGuest in $VMGuests) {
    $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
    $spec.changeVersion = $VMGuest.ExtensionData.Config.ChangeVersion
    $spec.tools = New-Object VMWare.Vim.ToolsConfigInfo
    
    $spec.tools.toolsUpgradePolicy = "upgradeAtPowerCycle"
    $spec.tools.syncTimeWithHost = $false

    # Apply the changes
    $MyVM = Get-View -Id $VMGuest.Id
    $MyVM.ReconfigVM_Task($spec)
}

So, there you have it.  Make sure all your VM hosts and guest are using the correct time, and are synchronised with a reliable time source.

Home Lab Network

networking-1

Introduction

If you have been following my Intel NUC As A ESXi Host series, you will notice that I have two networks, one called “Live” and one “Private”

 

Networks

Live

My “Live” network is everything that can potentially contact the internet: laptops, servers, printers, etc.  This network is on the 192.168.xxx.yyy/24 range – like almost everyone’s home network.

Private

The “Private” network is the one where most of my home lab virtual machines live.  They don’t have internet access, and in fact, can’t interact with anything on the “Live” network.  Their IP range is 10.1.1.xxx/24.

One of the reasons for this split, is to make sure anything I do on my home lab servers will not affect my home equipment.  For example, changing DNS entries, forcing group policies to computers, etc.

Management Network (VMware)

The Management Network seen in the screen shot above, is the management IP address of your ESXi host.  If you have a host with more than one NIC card (and really you should if you can), then this IP address is the one you use to connect to your host.

 

 Servers On Both Networks

I do have a few virtual servers that belong on both networks.  For example, my PRTG Network Monitor server can monitor both my “Live” and “Private” networks.  My Windows Server Update Services (WSUS) server can provide updates to both networks.

 

Adding More Networks

You can have as many networks as you like, I just need two for my purposes, but you can create more.  To do this, follow the steps below…

Since my NUCs only have one network port, I am using a vSphere Standard Switch, not a vSphere Distributed Switch.  Because of this, you will need to replicate the steps below on to each of your hosts…

  1. Open the vSphere Client and connect to your host (if you have one, or your vCenter Server if you have one of those),
  2. Click the Configuration tab, then click Networking on the left menu,
  3. If you used my automatic configuration script, you’ll see something like the screen shot above,
  4. Click the Properties link above the network card, not the one top right,

esx-networking-1

  1. Click Add and follow the wizard selecting the options below…
    • Connection Types : Virtual Machine
    • Network Label : [whatever name you want to have]
    • VLAN ID : [a number between 2 and 4094]

esx-networking-2   esx-networking-3   esx-networking-4

  1. Click Close when you have added all the networks you want.

When you create a new virtual machine, or edit the properties of one, you will now have all your networking options listed for you…

esx-networking-5

 

IP Range

Once you have created your networks, you can use whatever IP range you want within them, they will be separated within your home lab.  As I mentioned above, my “Private” network is using the 10.1.1.xxx/24 range, and for that I have a DHCP server handing out addresses just for that network.  More on my virtual machine setup in a future blog post.

 

VLANs

Before you start using VLANs between hosts (if you have more than one) on your network, make sure that you have a managed switch that can handle VLANs.  If you don’t have a managed switch, you won’t be able to them.  A managed switch is one that has it’s own IP address that you can login to.  The managed switch I have is a HP 1810-8G (J9802A).  This allows me to not only enable and use VLANs, but also IEEE 802.3ad Link Aggregation for my Synology NAS.

If you do have more than one NUC, you will need to make sure they are tagged in your switch to handle the VLANs.  On my particular switch..

  1. Go into VLANs > VLAN Configuration,
  2. Tick the Create VLAN box, and enter the VLAN ID number (as we did above),
  3. Click Apply,
  4. Go to VLANs > VLAN Configuration,
  5. Select the correct VLAN from the small drop down list,
  6. Select each of the ports your NUCs are plugged into, making sure they have the “T” mark to designate Tagged.
  7. Click Apply.

And that’s it, your virtual machines should now be able to talk to each other across hosts on whatever VLAN and IP range you configure for them.

Intel NUC As A ESXi Host (part 7)

Introduction

Now that ESXi has been installed, and you can manage it will the vCenter client, you will notice that your ESXi host is in maintenance mode, as it should be.  We set this at the end of the automatic script.

 

Table Of Contents

This will be a multi-part post, as it covers a lot of topics…

  1. NUC Specs, and build information,
  2. Building a custom ESX image that includes drivers for the NUC’s network card and SSD,
  3. DHCP, PXE booting and automatic installation scripts,
  4. Synology configuration for VMware datastores,
  5. Networking, VLANs and getting it to work,
  6. Installing ESXi,
  7. Any other configuration,
  8. Final (random) thoughts.

 

Other Configuration

You should also notice that the Storage and Network sections are also populated correctly with the settings you configured in the automated script.  You may need to rename your storage datastores if this is the first ESXi host.  The will have long “weird” names that are no use to anyone in a home lab.

Host Cache

Since I am using a 30Gb drive, and the ESXi installation only requires about 10Gb, I can use the remaining space as a host cache.  This is useful as it may help speed up your virtual machines.  To learn more about Host Caches, check out the official VMware documentation, and this great post by Duncan Epping at Yellow-Bricks.

Duncan’s Blog also gives the instructions for enabling the host cache, but I’ll give them here too, with screen shots of my system.

  1. Select your host (if you have more than one, you will need to do this on each one in turn),
  2. Select the Configuration tab along the top of the main section,
  3. Down the left hand menu, near the bottom, select Host Cache Configuration,
  4. You should see only one drive listed in the SSD Datastores list.  Notice my free space says 19.60GB,

HostCache-2

  1. Right click on the drive, and select Properties,
  2. In the next window, tick the box labeled “Allocate space for host cache“, and select the top radio button “Use maximum available space (19 GB)“.  Click OK when done.

HostCache-3

  1. The ESXi host will then create the host cache files required.  They are all 1Gb files, so in my case 19 of them will be created.  Once done, the host cache space and free space columns will change to show the correct values…

HostCache-4

If you then browse to the local datastore, you will see that there are indeed several files, all 1Gb in size…

HostCache-5

 

 

There will be more configuration settings coming soon.!

Intel NUC As A ESXi Host (part 8)

Introduction

We are all done, ESXi has been installed and is ready for you to create as many virtual machines as the little NUC powerhouse can handle.

 

Table Of Contents

This will be a multi-part post, as it covers a lot of topics…

  1. NUC Specs, and build information,
  2. Building a custom ESX image that includes drivers for the NUC’s network card and SSD,
  3. DHCP, PXE booting and automatic installation scripts,
  4. Synology configuration for VMware datastores,
  5. Networking, VLANs and getting it to work,
  6. Installing ESXi
  7. Any other configuration,
  8. Final (random) thoughts.

 

Final (random) thoughts

If you have more than one NUC, and want to setup a vCenter server to learn about clustering, I suggest you follow the amazing blog series by Derek Seaman here.  It has a huge amount of information on installing every single aspect of vCenter in such depth that even novices can follow.

 

If you are into scripting, and want to learn how to script your way around ESXi and vCenter, check out William Lam’s excellent blog.

Intel NUC As A ESXi Host (part 6)

Introduction

We are almost done, and now ready to finally install ESXi on our NUCs

 

Table Of Contents

This will be a multi-part post, as it covers a lot of topics…

  1. NUC Specs, and build information,
  2. Building a custom ESX image that includes drivers for the NUC’s network card and SSD,
  3. DHCP, PXE booting and automatic installation scripts,
  4. Synology configuration for VMware datastores,
  5. Networking, VLANs and getting it to work,
  6. Installing ESXi,
  7. Any other configuration,
  8. Final (random) thoughts.

 

ESXi Installation

Now that you have everything setup:

  • NUCs built,
  • DHCP ready,
  • TFTP and web server configured,
  • iSCSI targets provisioned,
  • Automatic installation script tweaked,

…you are ready to turn on your NUCs

This should be the easy bit, just power on your NUCs, and let the technology do the hard work.  It should boot up and immediately look for a DHCP address and server to boot from.

esx-install-1

 After a quick menu option, the installer will take over.  Don’t press anything – it’s all automatic remember  🙂

esx-install-2

 The installer will take about 10-20 minutes, depending on the model of NUC, network speeds, angle of the sun…

esx-install-3

 The installer will then change screens and start processing it’s various modules.  Below it’s loading the e1000 network driver.

esx-install-4

After loading all it’s modules, it will then run though part of the automatic script we created.

Various text messages will appear, including…

  • Preliminary checks,
  • Partitioning disk for ESXi,
  • Writing syslinux bootloader,
  • Writing binary to boot partition,
  • Writing GUIDs to the bootbanks,
  • Caching the required files for ESXi,
  • Configuring network settings,
  • Writing the first-boot scripts.

Once complete, the following reboot message will appear.  It will automatically reboot in about 10 seconds or so…

esx-install-5

ESXi will then load up and run the rest of the configuration script, rebooting again at the end…

esx-install-6

After the reboot, it will load all its modules.  Mine pauses for a while when loading the software iSCSI adapter.  This is because it will scan the adapter bus looking for available storage.  Once done though, you will be presented with a screen similar to below.  ESXi is now ready to be used…

esx-install-7

 

vSphere Client

Now that ESXi has been successfully installed, we can finish any other configuration that needs to be done.  If you don’t have the vSphere client installed, you can get it from your newly installed ESXi host.

Open your favourite web browser and go to the IP address (or host name) you gave your host.

esx-getting-started

Click the link middle-left to Download vSphere Client, and install it.  Once installed, open it and connect to your new host.  When the certificate warning appears, tick the checkbox, and click Ignore.

Congratulations, you have your first (or another) ESXi host.!

Intel NUC As A ESXi Host (part 5)

Introduction

Networking is an essential part of setting up your homelab.  Unfortunately, the NUCs only have one network card.  We can get around this limitation though with the use of VLANs.

 

Table Of Contents

This will be a multi-part post, as it covers a lot of topics…

  1. NUC Specs, and build information,
  2. Building a custom ESX image that includes drivers for the NUC’s network card and SSD,
  3. DHCP, PXE booting and automatic installation scripts,
  4. Synology configuration for VMware datastores,
  5. Networking, VLANs and getting it to work,
  6. Installing ESXi
  7. Any other configuration,
  8. Final (random) thoughts.

 

Networking and VLANs

With the NUCs only having one network card, things can get a little tricky, especially since I wanted separate live and private networks.  However, it can be done, and it works well enough for me and in a home lab environment.  One network card should never be used in a production environment.

My automatic configuration script creates the networks for me, however the basic layout is as below…

VMware Networking

My Private network is using the VLAN 42, my Live network has no VLAN.  The Management Network is on the IP address we gave to the NUC earlier in Part 3.  Check out the configuration script if you want your own settings.