Photon OS vCenter 6.5 deleting EAM folders in /tmp

To anyone that runs into an issue where hosts fail to get prepared with VXLAN by NSX, hopefully this post will help you out. This specific issue a very wise colleague(Mr.Sage) found, is when EAM(ESX Agent Manager) folders get deleted within the /tmp directory in the Photon OS 6.5 vCenter and that causes your hosts to not get prepared by NSX with VXLAN until a workaround is put in place or your restart EAM.

The good news is that there is a workaround(Please note I’m not expert on this and implementing this is done at your own risk

  1. First, as noted above you can simply restart EAM. Seems easy enough but how often do you reboot a host and how often do you really want to restart EAM?
    1. If you do want to restart EAM you can simply use the following command to check the status/stop/start vmware-eam

# Use this to check the status of EAM and simply change the "--status" to "--start" or "--stop"

service-control --status vmware-eam

  1. The other work around is to create a new file under this directory: /usr/lib/tmpfiles.d named tmp-eam.conf with the following contents:

# Exclude the following for EAM service
x /tmp/eam*

The above would allow the eam files to stay around until the system is rebooted. Once the vCenter is rebooted EAM would be restarted anyway and the files would be recreated.

Hope this helps and ping me with any feedback or questions

NSX API tips and guides

Lately I’ve been working more with the API for multiple versions of NSX with my colleague @VirtSouthWest. Here are a couple of of API calls that we have been using which are something I’d like to keep track of for future configurations and hope they help someone else:

To get started you need a REST API Client/plugin, here is one I use that works with FireFox – RESTCLIENT

Once you have that installed you are ready connect to your NSX manager. If you have a self signed cert you may need to go to the NSX Manager and accept the “not secure connection”. That is something good to check if you get a response like the one below:

api-auth-fail

Once you accept the security warning you and ensure you have the correct Authorization and Header in place you should get a 200 OK response as shown below:

api-auth-200ok

Here is a sample configuration of what you would send to a NSX manager API to configure Syslog. Make sure to specify the protocol TCP/UDP) and which port you have your syslog configured on, the standard being 514.

<syslogserver>
<syslogServer>Syslog-Server-FQDN/IP</syslogServer>
<port>514</port> - Port Configured on your Syslog Server
<protocol>UDP</protocol> - TCP/UDP
</syslogserver>

Here is a sample configuration of what you would send to a NSX manager API to configure NTP, you can configure 2 NTP servers using IP or FQDN which is great for redundancy.


<timeSettings>
<ntpServer>
<string>NTPServer-IP1</string><string>time1.google.com</string> - You can configure 2 NTP Servers
</ntpServer>
<timezone>UTC</timezone>
</timeSettings>

From the limited experince I have the backups are small ranging from 10-40MB.

Please note that once they reach their destination you configure they stay there and NSX does not currently clean up the backups. Meaning if you configure a backup job to run daily, after 1 year you will have 365 backups. This can take a while to load on the backup/restore screen. Please configure a job on the destination end to cleanup the backup jobs as needed. NSX will reflect these backups being gone and the list will be come shorter/load faster.

Here is a sample configuration of what you would send to a NSX manager API to configure scheduled Backups. In the example I have the time scheduled for 19:50 and for each manager you can configure the backup time. I have mine set to be staggered every 5 minutes.

Replace the following fields(Snip from the API Guide 6.2 below):

transferProtocol: FTP, SFTP

frequency: weekly, daily, hourly

dayOfWeek: SUNDAY, MONDAY, …., SATURDAY(Not in my example below)

Hour of Day: [0 ‐ 24 [  Minute of hour: [0 ‐ 60 [

Exclude Tables: AUDIT_LOG, SYSTEM_EVENTS, FLOW_RECORDS

The tables specified in the excludeTables parameter are not backed up.

<backupRestoreSettings>
<ftpSettings>
<transferProtocol>FTP</transferProtocol>
<hostNameIPAddress>Backup-Destination/IP-Address</hostNameIPAddress>
<port>21</port>
<userName>FTPUSER</userName><password>Password-for-FTPUSER</password>
<passPhrase>passPhrase</passPhrase> - For the backup file to restore
<backupDirectory>NSXBackupDir/</backupDirectory>
<filenamePrefix>NSX-Manager1-</filenamePrefix>
<passiveMode>true</passiveMode>
<useEPRT>false</useEPRT>
<useEPSV>true</useEPSV>
</ftpSettings>
<backupFrequency>
<frequency>DAILY</frequency>
<hourOfDay>19</hourOfDay>
<minuteOfHour>50</minuteOfHour>
</backupFrequency>
<excludeTables>
<excludeTable>AUDIT_LOGS</excludeTable>
<excludeTable>SYSTEM_EVENTS</excludeTable>
</excludeTables>
</backupRestoreSettings>

There are many of other things you can do via the NSX API and the above are just some some calls to get started. You can create controllers, controller backups, edges, etc.. Please review the guides below for the version you have.

API Guides link for different versions:

NSX 6.0.4 Guide

NSX 6.2 Guide

VCIX-NV

VMW-LGO-CERT-IMPLMT-EXPRT-NTWRK-VIRT_K

A few days ago I sat the VCIX-NV exam. This was not the first time that I had taken this exam but it was the last as I did manage to pass ! I will try and break down what I used to study for this exam and how my experience was.

To start, I do not work extensively with vSphere/NSX networking daily as far as installing/configuration goes but I do get a good amount of exposure to troubleshooting these areas. I was able to get some more hands on experience and some great breakdown thanks to my colleagues @Virtsouthwest(Mike A) and @Tompkins_23(John T).

Resources used:

Overall there are many resources out there for this exam and I will list a few that I found very helpful in preparing.

Jason Nash’s pluralsight courses found here were fantastic. I have always found pluralsight courses helpful for learning something new or getting more in depth into things. There are two courses, an intro course and a NSX Network Services course.  If you have a subscription these are a must watch.

Iwan Hoogendoorn’s set of NSX videos found on YouTube are a great breakdown in video form of the blueprint for this exam. HUGE thank you to Iwan for this content since it seems if you know all of the points in the VMware exam blueprint you will do just fine.

Martijn Smit’s blog LostDomain has a downloadable guide that breaks down the VCIX blueprint in detail with screenshots(I really like screenshots). Again it is vital to know each point in the blueprint for VMware exams. This guide is a perfect resource for knowing each point.

Exam experience:

Overall I had some issues much like others that have written about this exam. There are just some bugs with the version of NSX used in the exam. Be sure that if you do have an issue you get some help ASAP. I ran into issues that I can’t go into but I ended up needing assistance which resulted in a fix. I was able to keep moving forward with my exam and I was still in a decent mood(keep calm if you run into issues). Moving past the issues, I thought the exam itself did a good job of covering a wide range of network items that one could expect to run into in the course of being a VMware Admin/NSX Admin/vNetwork Admin. As I said above, know and be able to complete each point in the blueprint!