NSX Backups failing “HTTP request time out”

“Server failed to respond. HTTP request time out.”

I’ve been seeing this more frequently and thought I’d post something about how to fix this error that may happen when you click on the “Backup & Restore” button on the home page of NSX. It can affect the list of backups from loading and the backup jobs from running altogether. As you can see below NSX failed to load all of the backups in the history:


I have seen in the past a limit of 100 backups listed in history before performance of NSX listing those can be affected. I will try to link official documentation as soon as I find it.

I went and checked how many backups I had and found….. 830!! Just slightly over the (assumed) recommended amount of 100…

[root@ifitisnotbroken-file1 NSX1]# ls -lath | wc -l
[root@ifitisnotbroken-file1 NSX1]#

To clean this up, I ran the following command which was take from here:

find /ifitisnotbrokenbackups/NSX/NSX1-* -mtime +20 -exec rm {} \;

It finds any file that is over 20 days old(the +20 part) and will remove them. After I ran that command I had 82 files left:

[root@ifitisnotbroken-file1 NSX1]# ls -lath | wc -l
[root@ifitisnotbroken-file1 NSX1]#

I wanted to lower the number of files/backups so I didn’t have to do the cleanup process as often. I ran the above command but changed the days to 10 and I was left with 43 files. I refreshed the backup page in NSX and it took less than 17 seconds to load and my backups started working without issue:


Hope this helps and please let me know if you have any questions!


Photon OS vCenter 6.5 deleting EAM folders in /tmp

To anyone that runs into an issue where hosts fail to get prepared with VXLAN by NSX, hopefully this post will help you out. This specific issue a very wise colleague(Mr.Sage) found, is when EAM(ESX Agent Manager) folders get deleted within the /tmp directory in the Photon OS 6.5 vCenter and that causes your hosts to not get prepared by NSX with VXLAN until a workaround is put in place or your restart EAM.

The good news is that there is a workaround(Please note I’m not expert on this and implementing this is done at your own risk

  1. First, as noted above you can simply restart EAM. Seems easy enough but how often do you reboot a host and how often do you really want to restart EAM?
    1. If you do want to restart EAM you can simply use the following command to check the status/stop/start vmware-eam

# Use this to check the status of EAM and simply change the "--status" to "--start" or "--stop"

service-control --status vmware-eam

  1. The other work around is to create a new file under this directory: /usr/lib/tmpfiles.d named tmp-eam.conf with the following contents:

# Exclude the following for EAM service
x /tmp/eam*

The above would allow the eam files to stay around until the system is rebooted. Once the vCenter is rebooted EAM would be restarted anyway and the files would be recreated.

Hope this helps and ping me with any feedback or questions

NSX API tips and guides

Lately I’ve been working more with the API for multiple versions of NSX with my colleague @VirtSouthWest. Here are a couple of of API calls that we have been using which are something I’d like to keep track of for future configurations and hope they help someone else:

To get started you need a REST API Client/plugin, here is one I use that works with FireFox – RESTCLIENT

Once you have that installed you are ready connect to your NSX manager. If you have a self signed cert you may need to go to the NSX Manager and accept the “not secure connection”. That is something good to check if you get a response like the one below:


Once you accept the security warning you and ensure you have the correct Authorization and Header in place you should get a 200 OK response as shown below:


Here is a sample configuration of what you would send to a NSX manager API to configure Syslog. Make sure to specify the protocol TCP/UDP) and which port you have your syslog configured on, the standard being 514.

<port>514</port> - Port Configured on your Syslog Server
<protocol>UDP</protocol> - TCP/UDP

Here is a sample configuration of what you would send to a NSX manager API to configure NTP, you can configure 2 NTP servers using IP or FQDN which is great for redundancy.

<string>NTPServer-IP1</string><string>time1.google.com</string> - You can configure 2 NTP Servers

From the limited experince I have the backups are small ranging from 10-40MB.

Please note that once they reach their destination you configure they stay there and NSX does not currently clean up the backups. Meaning if you configure a backup job to run daily, after 1 year you will have 365 backups. This can take a while to load on the backup/restore screen. Please configure a job on the destination end to cleanup the backup jobs as needed. NSX will reflect these backups being gone and the list will be come shorter/load faster.

Here is a sample configuration of what you would send to a NSX manager API to configure scheduled Backups. In the example I have the time scheduled for 19:50 and for each manager you can configure the backup time. I have mine set to be staggered every 5 minutes.

Replace the following fields(Snip from the API Guide 6.2 below):

transferProtocol: FTP, SFTP

frequency: weekly, daily, hourly

dayOfWeek: SUNDAY, MONDAY, …., SATURDAY(Not in my example below)

Hour of Day: [0 ‐ 24 [  Minute of hour: [0 ‐ 60 [


The tables specified in the excludeTables parameter are not backed up.

<passPhrase>passPhrase</passPhrase> - For the backup file to restore

There are many of other things you can do via the NSX API and the above are just some some calls to get started. You can create controllers, controller backups, edges, etc.. Please review the guides below for the version you have.

API Guides link for different versions:

NSX 6.0.4 Guide

NSX 6.2 Guide



A few days ago I sat the VCIX-NV exam. This was not the first time that I had taken this exam but it was the last as I did manage to pass ! I will try and break down what I used to study for this exam and how my experience was.

To start, I do not work extensively with vSphere/NSX networking daily as far as installing/configuration goes but I do get a good amount of exposure to troubleshooting these areas. I was able to get some more hands on experience and some great breakdown thanks to my colleagues @Virtsouthwest(Mike A) and @Tompkins_23(John T).

Resources used:

Overall there are many resources out there for this exam and I will list a few that I found very helpful in preparing.

Jason Nash’s pluralsight courses found here were fantastic. I have always found pluralsight courses helpful for learning something new or getting more in depth into things. There are two courses, an intro course and a NSX Network Services course.  If you have a subscription these are a must watch.

Iwan Hoogendoorn’s set of NSX videos found on YouTube are a great breakdown in video form of the blueprint for this exam. HUGE thank you to Iwan for this content since it seems if you know all of the points in the VMware exam blueprint you will do just fine.

Martijn Smit’s blog LostDomain has a downloadable guide that breaks down the VCIX blueprint in detail with screenshots(I really like screenshots). Again it is vital to know each point in the blueprint for VMware exams. This guide is a perfect resource for knowing each point.

Exam experience:

Overall I had some issues much like others that have written about this exam. There are just some bugs with the version of NSX used in the exam. Be sure that if you do have an issue you get some help ASAP. I ran into issues that I can’t go into but I ended up needing assistance which resulted in a fix. I was able to keep moving forward with my exam and I was still in a decent mood(keep calm if you run into issues). Moving past the issues, I thought the exam itself did a good job of covering a wide range of network items that one could expect to run into in the course of being a VMware Admin/NSX Admin/vNetwork Admin. As I said above, know and be able to complete each point in the blueprint!