Ohio VMUG UserCon presentation: Upgrade to 6.5 with Zero Downtime

A little while ago I was able to fill in for a colleague of mine(@TomRalph) and present on his original VMworld session “Upgrading to vSphere 6.5 with Zero Downtime” at the Ohio VMUG/UserConference. It was a fantastic experience and my first time presenting at a VMUG/UserConference.

Being my first time presenting at a VMUG I was a bit nervous. I had to present twice that day and was lucky enough that another colleague of mine, Simon was presenting there as well and gave me plenty of pointers before and after my presentation, for which I am incredibly grateful!

Overall it was an amazing experience and if you haven’t had a chance to present and share your knowledge it is well worth it. Everyone involved was amazing to work with and encouraging to give back to the vcommunity.  Everyone wants you to succeed up there and to hear your experiences and story!

Advertisements

Host Profiles and the CLI Part 2

Building off of Part 1 , I forgot to add how to associate all hosts to the specified profile with the “associateonly” command:

PowerCLI C:\ Get-VMHost | Invoke-VMHostProfile -AssociateOnly -profile $profile

Now to test compliance against the hosts:

Test-VMHostProfileCompliance -VMHost *

Or against the profile specified:

Test-VMHostProfileCompliance -Profile SpecificNameHere

A failure would look similar to:

PowerCLI C:\> Test-VMHostProfileCompliance -VMHost ESXi-perfect | FL

VMHostId : HostSystem-host-20
VMHost : ESXi-perfect
VMHostUid : /VIServer=bbazan@ifitisnotbroken.com:443/VMHost=HostSystem-host-20/
VMHostProfileId : HostProfile-hostprofile-1
VMHostProfile : SpecificNameHere
VMHostProfileUid : /VIServer=bbazan@ifitisnotbrokencom:443/VMHostProfile=HostProfile-hostprofile-1/
IncomplianceElementList : {service_serviceProfile_ServiceConfigProfile-ProfExpression:Service TSM-SSH doesn't meet the running status False}
ExtensionData : VMware.Vim.ComplianceResult
Client : VMware.VimAutomation.ViCore.Impl.V1.VimClient

PowerCLI C:\>

The above shows that TSM-SSH doesn’t meet the running status False, meaning the Host Profile wants SSH off but is currently on. To make the host compliant to the Host Profile use the following command switching out the variable for the associated cluster/ set of hosts:

First, check that your host is in Maintenance Mode (and if not, put it in Maintenance Mode):

 set-VMHost -vmhost ESXi-perfect -State Maintenance

This will apply the Host Profile and then test compliance:

Invoke-VMHostProfile $vmhost | Test-VMHostProfileCompliance $vmhost

If the Host Profile was applied without any issues you will have no output from the check like this:

PowerCLI C:\> Test-VMHostProfileCompliance -VMHost ESXi-perfect | FL

PowerCLI C:\>

Now you can take the host out of Maintenance Mode!

set-VMHost -vmhost ESXi-perfect -State Connected

These can be modified to put a series of hosts into Maintenance Mode and apply the profile depending on your use case!

 

 

Host Profiles and the CLI Part 1

Depending on your licensing level you might want to create a Host Profile. This post is to show how to do it from the CLI. For those of you not familiar with Host Profiles you can read up here. When you have a host configured to a standard you are happy with you can use it as a reference host to “capture” those settings you just spent hours configuring :). Once you do that you will be able to apply it to other hosts and even setup a scheduled job to check of their compliance to make sure there hasn’t been any drift in configuration.

Create a Host Profile from the CLI:

 PowerCLI C:\ New-VMHostProfile -Name SpecificNameHere -ReferenceHost ESXi-perfect -Description "This is for testing compliance in the first vCenter" 

SpecificNameHere: some sort of descriptive name so you know what the Host Profile is testing against or its basic use

ESXi-perfect: The IP or Name of your reference host that is listed in vCenter that you  are now ready to capture its configuration.

Description: I find if people do not name their host profiles something useful they forget what they are for so a description might help you figure out why you made this in the first place.

If you were successful in your command you will now have a host profile!

PowerCLI C:\ New-VMHostProfile -Name SpecificNameHere -ReferenceHost ESXi-perfect -Description "This is for testing compliance in the first vCenter"

Name Description
---- -----------
SpecificNameHere This is for testing complia...

From there you can list out the details to make sure you selected the correct reference host.

PowerCLI C:\ Get-VMHostProfile | FL

ServerId : /VIServer=ifitisnotbroken\bbazan@FirstvCenter:443/
Server : FirstvCenter.ifitisnotbroken.com
Description : This is for testing compliance in the first vCenter
ReferenceHostId : HostSystem-host-##
ReferenceHost : ESXi-perfect.ifitisnotbroken.com
Name : SpecificNameHere
ExtensionData : VMware.Vim.HostProfile
Id : HostProfile-hostprofile-201
Uid : /VIServer=ifitisnotbroken\bbazan@FirstvCenter:443/:443/VMHostProfile=HostProfile-hostprofile-201/
Client : VMware.VimAutomation.ViCore.Impl.V1.VimClient
 

Now the trick is to attach it to other hosts and check their compliance against that Host Profile and remediate. You can attach the Host Profile to a Cluster and/or specific hosts by using these commands:

PowerCLI C:\ Invoke-VMHostProfile -AssociateOnly -Entity $cluster -Profile $profile

$cluster: the cluster you are wanting to associate this Host Profile to

$vmhost: the hosts you are wanting to associate this Host Profile to

$profile: the Host Profile you just created 🙂

At this point if you are worried about your Host Profile being deleted you can simply export a copy.

 PowerCLI C:\ Export-VMHostProfile c:\ -Profile SpecificNameHere 

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 12/22/2017 11:35 AM 1378229 SpecificNameHere.vpf

PowerCLI C:\

If you are wondering what I’m referencing for all of these magical commands it is a variety of docs including this from @vmwarecode code.vmware.com

Now we’ve got a Host Profile, assigned it to a host or cluster and exported a copy just in case.

Part 2 will cover checking compliance and remediating!

 

 

 

 

vCenter Host Profile compliance iSCSI Configuration error “Could not select an iSCSI HBA for the profile instance”

Today I ran into an issue that I could only find a partial answer to on the internet. My question was how to “remediate” this host profile failure in vCenter:

“Could not select an iSCSI HBA for the profile instance: 08756bda103420a7485c6cb8c2cb79b4cacbbab40be1830e9e06bfefaef25cc3. One of following may be true: (1) Some of the required user-input host customization settings are missing for the host (2) The system does not have a matching hardware for the given ‘Initiator Selection Policy'”

I attempted to un-check these adapters causing the host profile compliance failures. That did not actually help. The compliance failure was still there.

I decided to take a backup of the host profile config just in case something went wrong and delete these from the profile. These are located in the host profile under “Storage configuration –> iSCSI Initiator Configuration –> Dependent Hardware iSCSI Adapters”. For my issue I had two that were not in use. I deleted the adapters from the profile, checked compliance and all was green!!

 

 

NSX Backups failing “HTTP request time out”

“Server failed to respond. HTTP request time out.”

I’ve been seeing this more frequently and thought I’d post something about how to fix this error that may happen when you click on the “Backup & Restore” button on the home page of NSX. It can affect the list of backups from loading and the backup jobs from running altogether. As you can see below NSX failed to load all of the backups in the history:

I have seen in the past a limit of 100 backups listed in history before performance of NSX listing those can be affected. I will try to link official documentation as soon as I find it.

I went and checked how many backups I had and found….. 830!! Just slightly over the (assumed) recommended amount of 100…

[root@ifitisnotbroken-file1 NSX1]# ls -lath | wc -l
830
[root@ifitisnotbroken-file1 NSX1]#

To clean this up, I ran the following command which was take from here:

find /ifitisnotbrokenbackups/NSX/NSX1-* -mtime +20 -exec rm {} \;

It finds any file that is over 20 days old(the +20 part) and will remove them. After I ran that command I had 82 files left:

[root@ifitisnotbroken-file1 NSX1]# ls -lath | wc -l
82
[root@ifitisnotbroken-file1 NSX1]#

I wanted to lower the number of files/backups so I didn’t have to do the cleanup process as often. I ran the above command but changed the days to 10 and I was left with 43 files. I refreshed the backup page in NSX and it took less than 17 seconds to load and my backups started working without issue:

Hope this helps and please let me know if you have any questions!

 

Fenced vApps are broken in vCloud Director 8.x with NSX 6.3.x

 

Updated with a fix!

 

I’m a huge fan of vCloud Director and the amazing product that NSX is; that being said this is not a knock against vCD or NSX but rather more of a “PSA”. Using vCloud Director and NSX is fantastic aside from this little bug/known issue. If you use fencing in your vCD 8.x environment and NSX 6.2.x do not upgrade just yet. Please see the notes here in a VMware KB about why upgrading to NSX 6.3.x in a vCD 8.x environment will break fenced vApps.

Here is a snip from the KB itself

In NSX 6.3.x Edge Gateways, a new route table is introduced (table 251) which is always looked up first for routes. The main table (table 254) is looked up only when the table 251 does not have a route. The issue occurs because the device routes (a /32 route for vApp VM) are auto-plumbed to the main table, whereas table 251 already has a default route in it. Therefore, since the table 251 already has a default route, the lookup never happens on the main table and hence the fenced vApp virtual machines lose connectivity.

VMware is still working on a fix internally and I’ll try to update this blog once I know more about the fix and when it is released.

 

***UPDATE***

 

Please see the following release doc stating the Fencing issue is fixed in 6.3.2

http://pubs.vmware.com/Release_Notes/en/nsx/6.3.2/releasenotes_nsx_vsphere_632.html

Enjoy!

Why get VMware Certified and keep it current?

Today I decided to take a step back and think why I got certified in the first place(way back in 2009). People have very different motivations for getting certified, mine was not related to monetary gain or advancement at my workplace when I first (started using haha – they are a bit like an addiction) took a VMware exam.  I worked for a very small company and I wanted a way to prove to myself that I knew what I was actually doing; also prove to my employer that I knew what I was doing!

Enter the VCP-4 exam, I felt this was the perfect way to show that I did know all of the features, such as how to configure ESX(i)/vCenter, which license level gave you which specific features and most importantly what all of those cool features did/could do.  Working for a small company I did not have the luxury of getting exams paid for sadly, so when I decided to take my exam I needed to make sure I was ready, had put the time in to study and find every resource possible to learn all that I could.

My first attempt I was very nervous and unsure even though I had spent 2 months going over things I thought I would need to know. I failed, I not only failed I bombed it. I did not let it get my down for long and after that I decided to start fresh and go over this “blueprint” thing that I had downloaded but did not pay (enough)attention to. I began reviewing it and looking over various breakdowns of the blueprint on blogs like this one from Simon Long. I took another 3 months to study, really take time to go over the blueprint, and feel comfortable with all topics. During that 3 months I spent time reading over VMware documentation that was relevant to the blueprint and learning everything I could.

I lined up another exam date and was ready to give it a go after months of study time. I was still nervous but felt much better about taking it this time. I had gained more confidence after putting in the study time and felt like I could really do this(now that did not mean that I didn’t get a little flustered whilst taking the exam). I passed this time with flying colors. I felt on top of the world after that.

Moving on from the first pass:

I really started reading more and more after passing that first exam. I setup a home lab and found many answers to my questions on the VMware community page, and on Twitter. The community that was around at that time was already really amazing as people were so helpful and eager to answer questions.

Did this actually help you or your career and why keep taking them?

So, why take these exams and put all of this time in? Well, again I started this to justify my knowledge to myself really. I kept taking these over the years as the versions changed and new exams came out; I wanted to push myself to learn more/new technologies and show it by passing these exams. By passing these exams I got an opportunity to interview for an amazing position. Part of why I got the position is I took the time to pursue these certs on my own and kept learning. It made a huge difference in my career passing these certs and moving into a role where I have had and still have tons of room to grow.

Looking back at this first exam I really learned what VMware was expecting and that you can’t ignore the blueprint!! The issues I have/had thinking about this one is that you had to memorize so many config min/max settings that many people found to be useless. Moving on from version 4 I feel that VMware has correct this issue and is testing for much more “real life” applicable knowledge from people attempting these tests. I also felt that it was a huge benefit to setup a home lab going forward with my career and future exams. Having taken many more exams since the VCP4 I really have a good process for prepping for the exam which includes reviewing the full blueprint. A fantastic example of covering the blueprint is what Mike Preston did on his blog  when he covered his 8 weeks of VCAP which I used for my first VCAP!

 

Never stop learning!