Host Profiles and the CLI Part 2

Building off of Part 1 , I forgot to add how to associate all hosts to the specified profile with the “associateonly” command:

PowerCLI C:\ Get-VMHost | Invoke-VMHostProfile -AssociateOnly -profile $profile

Now to test compliance against the hosts:

Test-VMHostProfileCompliance -VMHost *

Or against the profile specified:

Test-VMHostProfileCompliance -Profile SpecificNameHere

A failure would look similar to:


PowerCLI C:\> Test-VMHostProfileCompliance -VMHost ESXi-perfect | FL

VMHostId : HostSystem-host-20
VMHost : ESXi-perfect
VMHostUid : /VIServer=bbazan@ifitisnotbroken.com:443/VMHost=HostSystem-host-20/
VMHostProfileId : HostProfile-hostprofile-1
VMHostProfile : SpecificNameHere
VMHostProfileUid : /VIServer=bbazan@ifitisnotbrokencom:443/VMHostProfile=HostProfile-hostprofile-1/
IncomplianceElementList : {service_serviceProfile_ServiceConfigProfile-ProfExpression:Service TSM-SSH doesn't meet the running status False}
ExtensionData : VMware.Vim.ComplianceResult
Client : VMware.VimAutomation.ViCore.Impl.V1.VimClient

PowerCLI C:\>

The above shows that TSM-SSH doesn’t meet the running status False, meaning the Host Profile wants SSH off but is currently on. To make the host compliant to the Host Profile use the following command switching out the variable for the associated cluster/ set of hosts:

First, check that your host is in Maintenance Mode (and if not, put it in Maintenance Mode):

 set-VMHost -vmhost ESXi-perfect -State Maintenance

This will apply the Host Profile and then test compliance:

Invoke-VMHostProfile $vmhost | Test-VMHostProfileCompliance $vmhost

If the Host Profile was applied without any issues you will have no output from the check like this:


PowerCLI C:\> Test-VMHostProfileCompliance -VMHost ESXi-perfect | FL

PowerCLI C:\>

Now you can take the host out of Maintenance Mode!

set-VMHost -vmhost ESXi-perfect -State Connected

These can be modified to put a series of hosts into Maintenance Mode and apply the profile depending on your use case!

 

 

Advertisements

Host Profiles and the CLI Part 1

Depending on your licensing level you might want to create a Host Profile. This post is to show how to do it from the CLI. For those of you not familiar with Host Profiles you can read up here. When you have a host configured to a standard you are happy with you can use it as a reference host to “capture” those settings you just spent hours configuring :). Once you do that you will be able to apply it to other hosts and even setup a scheduled job to check of their compliance to make sure there hasn’t been any drift in configuration.

Create a Host Profile from the CLI:

 PowerCLI C:\ New-VMHostProfile -Name SpecificNameHere -ReferenceHost ESXi-perfect -Description "This is for testing compliance in the first vCenter" 

SpecificNameHere: some sort of descriptive name so you know what the Host Profile is testing against or its basic use

ESXi-perfect: The IP or Name of your reference host that is listed in vCenter that you  are now ready to capture its configuration.

Description: I find if people do not name their host profiles something useful they forget what they are for so a description might help you figure out why you made this in the first place.

If you were successful in your command you will now have a host profile!

PowerCLI C:\ New-VMHostProfile -Name SpecificNameHere -ReferenceHost ESXi-perfect -Description "This is for testing compliance in the first vCenter"

Name Description
---- -----------
SpecificNameHere This is for testing complia...

From there you can list out the details to make sure you selected the correct reference host.

PowerCLI C:\ Get-VMHostProfile | FL

ServerId : /VIServer=ifitisnotbroken\bbazan@FirstvCenter:443/
Server : FirstvCenter.ifitisnotbroken.com
Description : This is for testing compliance in the first vCenter
ReferenceHostId : HostSystem-host-##
ReferenceHost : ESXi-perfect.ifitisnotbroken.com
Name : SpecificNameHere
ExtensionData : VMware.Vim.HostProfile
Id : HostProfile-hostprofile-201
Uid : /VIServer=ifitisnotbroken\bbazan@FirstvCenter:443/:443/VMHostProfile=HostProfile-hostprofile-201/
Client : VMware.VimAutomation.ViCore.Impl.V1.VimClient
 

Now the trick is to attach it to other hosts and check their compliance against that Host Profile and remediate. You can attach the Host Profile to a Cluster and/or specific hosts by using these commands:

PowerCLI C:\ Invoke-VMHostProfile -AssociateOnly -Entity $cluster -Profile $profile

$cluster: the cluster you are wanting to associate this Host Profile to

$vmhost: the hosts you are wanting to associate this Host Profile to

$profile: the Host Profile you just created 🙂

At this point if you are worried about your Host Profile being deleted you can simply export a copy.

 PowerCLI C:\ Export-VMHostProfile c:\ -Profile SpecificNameHere 

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 12/22/2017 11:35 AM 1378229 SpecificNameHere.vpf

PowerCLI C:\

If you are wondering what I’m referencing for all of these magical commands it is a variety of docs including this from @vmwarecode code.vmware.com

Now we’ve got a Host Profile, assigned it to a host or cluster and exported a copy just in case.

Part 2 will cover checking compliance and remediating!

 

 

 

 

vCenter Host Profile compliance iSCSI Configuration error “Could not select an iSCSI HBA for the profile instance”

Today I ran into an issue that I could only find a partial answer to on the internet. My question was how to “remediate” this host profile failure in vCenter:

“Could not select an iSCSI HBA for the profile instance: 08756bda103420a7485c6cb8c2cb79b4cacbbab40be1830e9e06bfefaef25cc3. One of following may be true: (1) Some of the required user-input host customization settings are missing for the host (2) The system does not have a matching hardware for the given ‘Initiator Selection Policy'”

I attempted to un-check these adapters causing the host profile compliance failures. That did not actually help. The compliance failure was still there.

I decided to take a backup of the host profile config just in case something went wrong and delete these from the profile. These are located in the host profile under “Storage configuration –> iSCSI Initiator Configuration –> Dependent Hardware iSCSI Adapters”. For my issue I had two that were not in use. I deleted the adapters from the profile, checked compliance and all was green!!

 

 

NSX Backups failing “HTTP request time out”

“Server failed to respond. HTTP request time out.”

I’ve been seeing this more frequently and thought I’d post something about how to fix this error that may happen when you click on the “Backup & Restore” button on the home page of NSX. It can affect the list of backups from loading and the backup jobs from running altogether. As you can see below NSX failed to load all of the backups in the history:

nsx-backup-issues-1.png

I have seen in the past a limit of 100 backups listed in history before performance of NSX listing those can be affected. I will try to link official documentation as soon as I find it.

I went and checked how many backups I had and found….. 830!! Just slightly over the (assumed) recommended amount of 100…

[root@ifitisnotbroken-file1 NSX1]# ls -lath | wc -l
830
[root@ifitisnotbroken-file1 NSX1]#

To clean this up, I ran the following command which was take from here:

find /ifitisnotbrokenbackups/NSX/NSX1-* -mtime +20 -exec rm {} \;

It finds any file that is over 20 days old(the +20 part) and will remove them. After I ran that command I had 82 files left:

[root@ifitisnotbroken-file1 NSX1]# ls -lath | wc -l
82
[root@ifitisnotbroken-file1 NSX1]#

I wanted to lower the number of files/backups so I didn’t have to do the cleanup process as often. I ran the above command but changed the days to 10 and I was left with 43 files. I refreshed the backup page in NSX and it took less than 17 seconds to load and my backups started working without issue:

nsx-backup-issues-2

Hope this helps and please let me know if you have any questions!

 

Why get VMware Certified and keep it current?

Today I decided to take a step back and think why I got certified in the first place(way back in 2009). People have very different motivations for getting certified, mine was not related to monetary gain or advancement at my workplace when I first (started using haha – they are a bit like an addiction) took a VMware exam.  I worked for a very small company and I wanted a way to prove to myself that I knew what I was actually doing; also prove to my employer that I knew what I was doing!

Enter the VCP-4 exam, I felt this was the perfect way to show that I did know all of the features, such as how to configure ESX(i)/vCenter, which license level gave you which specific features and most importantly what all of those cool features did/could do.  Working for a small company I did not have the luxury of getting exams paid for sadly, so when I decided to take my exam I needed to make sure I was ready, had put the time in to study and find every resource possible to learn all that I could.

My first attempt I was very nervous and unsure even though I had spent 2 months going over things I thought I would need to know. I failed, I not only failed I bombed it. I did not let it get my down for long and after that I decided to start fresh and go over this “blueprint” thing that I had downloaded but did not pay (enough)attention to. I began reviewing it and looking over various breakdowns of the blueprint on blogs like this one from Simon Long. I took another 3 months to study, really take time to go over the blueprint, and feel comfortable with all topics. During that 3 months I spent time reading over VMware documentation that was relevant to the blueprint and learning everything I could.

I lined up another exam date and was ready to give it a go after months of study time. I was still nervous but felt much better about taking it this time. I had gained more confidence after putting in the study time and felt like I could really do this(now that did not mean that I didn’t get a little flustered whilst taking the exam). I passed this time with flying colors. I felt on top of the world after that.

Moving on from the first pass:

I really started reading more and more after passing that first exam. I setup a home lab and found many answers to my questions on the VMware community page, and on Twitter. The community that was around at that time was already really amazing as people were so helpful and eager to answer questions.

Did this actually help you or your career and why keep taking them?

So, why take these exams and put all of this time in? Well, again I started this to justify my knowledge to myself really. I kept taking these over the years as the versions changed and new exams came out; I wanted to push myself to learn more/new technologies and show it by passing these exams. By passing these exams I got an opportunity to interview for an amazing position. Part of why I got the position is I took the time to pursue these certs on my own and kept learning. It made a huge difference in my career passing these certs and moving into a role where I have had and still have tons of room to grow.

Looking back at this first exam I really learned what VMware was expecting and that you can’t ignore the blueprint!! The issues I have/had thinking about this one is that you had to memorize so many config min/max settings that many people found to be useless. Moving on from version 4 I feel that VMware has correct this issue and is testing for much more “real life” applicable knowledge from people attempting these tests. I also felt that it was a huge benefit to setup a home lab going forward with my career and future exams. Having taken many more exams since the VCP4 I really have a good process for prepping for the exam which includes reviewing the full blueprint. A fantastic example of covering the blueprint is what Mike Preston did on his blog  when he covered his 8 weeks of VCAP which I used for my first VCAP!

 

Never stop learning!

 

Photon OS vCenter 6.5 deleting EAM folders in /tmp

To anyone that runs into an issue where hosts fail to get prepared with VXLAN by NSX, hopefully this post will help you out. This specific issue a very wise colleague(Mr.Sage) found, is when EAM(ESX Agent Manager) folders get deleted within the /tmp directory in the Photon OS 6.5 vCenter and that causes your hosts to not get prepared by NSX with VXLAN until a workaround is put in place or your restart EAM.

The good news is that there is a workaround(Please note I’m not expert on this and implementing this is done at your own risk

  1. First, as noted above you can simply restart EAM. Seems easy enough but how often do you reboot a host and how often do you really want to restart EAM?
    1. If you do want to restart EAM you can simply use the following command to check the status/stop/start vmware-eam

# Use this to check the status of EAM and simply change the "--status" to "--start" or "--stop"

service-control --status vmware-eam

  1. The other work around is to create a new file under this directory: /usr/lib/tmpfiles.d named tmp-eam.conf with the following contents:

# Exclude the following for EAM service
x /tmp/eam*

The above would allow the eam files to stay around until the system is rebooted. Once the vCenter is rebooted EAM would be restarted anyway and the files would be recreated.

Hope this helps and ping me with any feedback or questions

NSX API tips and guides

Lately I’ve been working more with the API for multiple versions of NSX with my colleague @VirtSouthWest. Here are a couple of of API calls that we have been using which are something I’d like to keep track of for future configurations and hope they help someone else:

To get started you need a REST API Client/plugin, here is one I use that works with FireFox – RESTCLIENT

Once you have that installed you are ready connect to your NSX manager. If you have a self signed cert you may need to go to the NSX Manager and accept the “not secure connection”. That is something good to check if you get a response like the one below:

api-auth-fail

Once you accept the security warning you and ensure you have the correct Authorization and Header in place you should get a 200 OK response as shown below:

api-auth-200ok

Here is a sample configuration of what you would send to a NSX manager API to configure Syslog. Make sure to specify the protocol TCP/UDP) and which port you have your syslog configured on, the standard being 514.

<syslogserver>
<syslogServer>Syslog-Server-FQDN/IP</syslogServer>
<port>514</port> - Port Configured on your Syslog Server
<protocol>UDP</protocol> - TCP/UDP
</syslogserver>

Here is a sample configuration of what you would send to a NSX manager API to configure NTP, you can configure 2 NTP servers using IP or FQDN which is great for redundancy.


<timeSettings>
<ntpServer>
<string>NTPServer-IP1</string><string>time1.google.com</string> - You can configure 2 NTP Servers
</ntpServer>
<timezone>UTC</timezone>
</timeSettings>

From the limited experince I have the backups are small ranging from 10-40MB.

Please note that once they reach their destination you configure they stay there and NSX does not currently clean up the backups. Meaning if you configure a backup job to run daily, after 1 year you will have 365 backups. This can take a while to load on the backup/restore screen. Please configure a job on the destination end to cleanup the backup jobs as needed. NSX will reflect these backups being gone and the list will be come shorter/load faster.

Here is a sample configuration of what you would send to a NSX manager API to configure scheduled Backups. In the example I have the time scheduled for 19:50 and for each manager you can configure the backup time. I have mine set to be staggered every 5 minutes.

Replace the following fields(Snip from the API Guide 6.2 below):

transferProtocol: FTP, SFTP

frequency: weekly, daily, hourly

dayOfWeek: SUNDAY, MONDAY, …., SATURDAY(Not in my example below)

Hour of Day: [0 ‐ 24 [  Minute of hour: [0 ‐ 60 [

Exclude Tables: AUDIT_LOG, SYSTEM_EVENTS, FLOW_RECORDS

The tables specified in the excludeTables parameter are not backed up.

<backupRestoreSettings>
<ftpSettings>
<transferProtocol>FTP</transferProtocol>
<hostNameIPAddress>Backup-Destination/IP-Address</hostNameIPAddress>
<port>21</port>
<userName>FTPUSER</userName><password>Password-for-FTPUSER</password>
<passPhrase>passPhrase</passPhrase> - For the backup file to restore
<backupDirectory>NSXBackupDir/</backupDirectory>
<filenamePrefix>NSX-Manager1-</filenamePrefix>
<passiveMode>true</passiveMode>
<useEPRT>false</useEPRT>
<useEPSV>true</useEPSV>
</ftpSettings>
<backupFrequency>
<frequency>DAILY</frequency>
<hourOfDay>19</hourOfDay>
<minuteOfHour>50</minuteOfHour>
</backupFrequency>
<excludeTables>
<excludeTable>AUDIT_LOGS</excludeTable>
<excludeTable>SYSTEM_EVENTS</excludeTable>
</excludeTables>
</backupRestoreSettings>

There are many of other things you can do via the NSX API and the above are just some some calls to get started. You can create controllers, controller backups, edges, etc.. Please review the guides below for the version you have.

API Guides link for different versions:

NSX 6.0.4 Guide

NSX 6.2 Guide