VMware Cloud Director: ExtraConfig Keys and NSX Unified Appliance

Have you encountered the error “Validation failed for the OVF file you provided: The OVF uses vmw:ExtraConfig elements which are not allowed by vCloud Director“? You might have noticed it prevents content from being uploaded or imported. Different appliances and a wide range of content require “ExtraConfig” keys in a VCD database (or imported via cell-management-tool). Reference to the commands for the cell-management-tool can be found here. Below, I’ve posted my team’s workaround and detailed some NSX related ExtraConfig keys.

**Disclaimer #1: use these DB modifications at your own risk and please take backups**

**Disclaimer #2: This workaround is highly unsupported**

Where do you find these settings? Opening the OVF file, a simple search for “ExtraConfig” will show what elements are being used. Example:

      <vmw:Config ovf:required="false" vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
      <vmw:ExtraConfig vmw:key="time.synchronize.continue" vmw:value="false"/>
      <vmw:ExtraConfig vmw:key="time.synchronize.restore" vmw:value="false"/>
      <vmw:ExtraConfig vmw:key="time.synchronize.resume.disk" vmw:value="false"/>
      <vmw:ExtraConfig vmw:key="time.synchronize.shrink" vmw:value="false"/>
      <vmw:ExtraConfig vmw:key="time.synchronize.tools.startup" vmw:value="false"/>
      <vmw:ExtraConfig vmw:key="ethernet0.rxDataRingEnabled" vmw:value="0"/>
      <vmw:ExtraConfig vmw:key="ethernet1.rxDataRingEnabled" vmw:value="0"/>
      <vmw:ExtraConfig vmw:key="ethernet2.rxDataRingEnabled" vmw:value="0"/>
      <vmw:ExtraConfig vmw:key="ethernet3.rxDataRingEnabled" vmw:value="0"/>
      <vmw:ExtraConfig vmw:key="isolation.tools.diskWiper.disable" vmw:value="true"/>
      <vmw:ExtraConfig vmw:key="isolation.tools.memSchedFakeSampleStats.disable" vmw:value="true"/>
      <vmw:ExtraConfig vmw:key="isolation.tools.diskShrink.disable" vmw:value="true"/>
      <vmw:ExtraConfig vmw:key="isolation.tools.vmxDnDVersionGet.disable" vmw:value="true"/>
      <vmw:ExtraConfig vmw:key="isolation.tools.unityActive.disable" vmw:value="true"/>
      <vmw:ExtraConfig vmw:key="isolation.tools.guestDnDVersionSet.disable" vmw:value="true"/>
      <vmw:ExtraConfig vmw:key="snapshot.maxSnapshots" vmw:value="0"/>
      <vmw:ExtraConfig vmw:key="RemoteDisplay.maxConnections" vmw:value="1"/>

Below is a small sample of how to insert these into the DB. A larger list can be found here. Please do research before blindly allowing every ExtraConfig key to be imported into Cloud Director.

INSERT into config (config_id, cat, name, value)
values

    --ExtraConfigs for nsx-unified-appliance
    (NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'ethernet0.rxDataRingEnabled', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'ethernet1.rxDataRingEnabled', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'ethernet2.rxDataRingEnabled', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'ethernet3.rxDataRingEnabled', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'isolation.tools.diskWiper.disable', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'isolation.tools.memSchedFakeSampleStats.disable', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'isolation.tools.diskShrink.disable', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'isolation.tools.vmxDnDVersionGet.disable', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'isolation.tools.unityActive.disable', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'isolation.tools.guestDnDVersionSet.disable', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'sched.mem.pin', 'true')
    ,(NEXTVAL('seq_config'), 'ExtraConfigWhitelist', 'svga.autodetect', 'true')
  ON CONFLICT (cat, name) DO UPDATE
  SET value = EXCLUDED.value;

NSX Unified Appliances within Cloud Director allow 0 snapshots for the NSX VM by default. Taking a snapshot will fail if this value is not changed:

<vmw:ExtraConfig vmw:key="snapshot.maxSnapshots" vmw:value="0"/>

The failure (and where to update the value):

VMware Cloud Director: How to consolidate VMs in a vApp Template

As a follow up to this post: vCloud Director: How to consolidate a VM with PowerCLI, I’m creating this one which is almost identical for consolidating all VMs in a vApp templates or individual VMs within a vApp Template with VMware Cloud Director aka VCD or vCD 😉

The feature to consolidate in the UI is exposed in 10.2 versions of VCD but not in earlier versions it seems.

The script found here vCD-consolidate-vapptemplate-vm.ps1 and takes the required input of Org name, Catalog name, vApp Template name. Optionally VM name. The VM name can be left blank and it will consolidate all VMs within the vApp Template.

The same process is followed for checking power state, as a powered on vApp Template will cause errors while attempting to consolidate.

Good luck and hopefully this is useful somewhere along the way!

vCloud Director: A general system error occurred: Exceeded the maximum number of permitted snapshots while capturing vApp to a catalog

During some troubleshooting this past week, the following error came up while trying to capture a vApp to a catalog within VCD:

An error occurred while taking a snapshot: Exceeded the maximum number of permitted snapshots.

Along with a more detailed version:

[ 108e6549-9f26-8e11-a55a-584120dd8ead ] “Cannot create snapshot of VM [vcId=UUID-HERE, moref=vm-6].”
 - Underlying system error:
An error occurred while taking a snapshot: Exceeded the maximum number of permitted snapshots.
vCenter Server task (moref: task-143) failed in vCenter Server ‘bblab-vc1’ (UUID-HERE).
 - An error occurred while taking a snapshot: Exceeded the maximum number of permitted snapshots.

This specific issue centered around a NSX VM that was being used within Cloud Director.
Before checking on that specific VM I found the following KB which confirmed my suspicions that a hard limit has been set on the VM itself.

To confirm this, login to the vCenter where this VM exists. Edit the settings of the VM, browse to “VM Options”, searching the advanced configuration parameters the snapshot.maxSnpshots was present and set to 0 which disables snapshots.

This was updated to a value that the user can decide on(more than 0 to allow snapshots) and from there the content was able to be captured once again

Directly from the KB(please note the CAUTION line and use at your own risk):

To resolve this issue:

1. Power down the virtual machine.
2. Right-click the virtual machine in the Web Client and click Edit Settings.
3. Click the VM Options tab, under Advanced, select the Edit Configuration option.
4. Search for a string name called snapshot.maxSnapshots and increase its value.
5. Click OK to save the changes.
6. Power on the virtual machine and create the required snapshot.

Caution: Only modify the snapshot.maxSnapshots parameter with the guidance of VMware technical support. Increasing the number of allowed snapshots beyond the maximum value of 32 is not supported.
Notes:

• If the snapshot.maxSnapshots value is not present, you may manually enter this configuration setting. In vSphere 6.7.x and 7.0.x, the setting is not listed by default.
• To disallow snapshots, set the snapshot.maxSnapshots value to 0.

vCloud Director: How to bypass SAML authentication for a tenant Org in the H5 UI

In the past tenants with Orgs in vCloud Director that were configured with SAML authentication were able to bypass the SAML authentication redirect by adding /login.jsp to the end of the Org URL. This would then allow a tenant to login with a local or LDAP account.

For example: https://bblab-vcd.com/cloud/org/testing-org/ would be changed to https://bblab-vcd.com/cloud/org/testing-org/login.jsp to allow local or LDAP authentication

Fast forward, there is no more FLEX/Flash UI and I could not find anything documented for the HTML5/H5 UI on how to do this. Trying to login to the Default organization URL(for example: https://bblab-vcd.com/tenant/testing-org/) would fail and could result in the following screen:

In the H5 UI, if the need arises to bypass the SAML authentication; /login could be added to the Default organization URL: https://bblab-vcd.com/tenant/testing-org/login which should redirect to something similar to this: https://bblab-vcd.com/login/login.jsp?service=tenant:testing-org&redirectTo=%2Ftenant%2Ftesting-org .Resulting in a much more pleasant page prompting for a User name/Password:

vCloud Director: HTML5 UI Login fails – “Failed to Start” error and fix

After upgrading a vCloud Director instance to 10.x, I’ve seen many logins fail with the following “Failed to Start” error message in the HTML5 UI only(**Note the FLEX UI is gone in 10.1 versions) This is covered in the following VMware KB as well: https://kb.vmware.com/s/article/75305

**Note this is found on vCD instances where the cells have been installed on linux, I’ve yet to test the appliance for the same**

Digging into this more, this error is caused by something called CORS filter not being updated with all values required. What is CORS and what values should be in CORS?

CORS stands for Cross-Origin Resource Sharing and is used by the H5 UI NOT Flex(flash) UI. The filter limits how each vCD endpoint can be accessed. In the configuration of the cells, if the CORS filter list doesn’t not contain the DNS, IP address, and shortname of each cell the login fails.

The workaround is a pretty quick and painless one. Run the following command on any cell in the vCD instance that is giving this error to retrieve the current CORS configuration:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n webapp.allowed.origins -l

The above will return something similar to this and in my case this is missing all 3 cell instances:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n webapp.allowed.origins -l ",10.1.1.1,http://10.1.1.3,http://10.1.1.2,http://10.1.1.1,https://10.1.1.2,https://10.1.1.1,https://10.1.1.3,https://vcd-bblab.com,http://vcd.bblab.com”

This list is comma separated, so simply adding in the missing values and making one small change to edit the values on the cell instead of listing them is all that is needed. To do that change the webapp.allowed.origins -l to a webapp.allowed.origins -v:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n webapp.allowed.origins -v "10.1.1.1,http://10.1.1.3,http://10.1.1.2,http://10.1.1.1,https://10.1.1.2,https://10.1.1.1,https://10.1.1.3,https://vcd-bblab.com,http://vcd.bblab.com,cell1.bblab.com,cell2.bblab.com,cell3.bblab.com,http://cell1.bblab.com,https://cell1.bblab.com,http://cell2.bblab.com,https://cell2.bblab.com,http://cell3.bblab.com,https://cell3.bblab.com”

Once the CORS values have been updated with the required information, logins will work once again without the need of restarting vCD cell services.

**Not supported, us at your own risk!!!**

These values are all stored in the vCloud Director database under the “config” table under “webapp.allowed.origins”

ESXi 7.0 Upgrade and Lifecycle Manager/VUM VIB dependencies

In my current position I’m working with upgrades at scale, and using  VUM/Lifecycle Manager (new to 7.0). Recently, I went to start the upgrade process on a number of hosts but ran into an issue pretty quick while trying to complete remediation. The exact error was this:

The upgrade has VIBs that are missing dependencies:

(List of 10 VIBs)

Remove the VIBs or use Image Builder to create a custom upgrade ISO image that contains the missing dependencies, and try to upgrade again.

It should be easy to search a way to massively remove these VIBs and continue the upgrade… nope. Enter this blog post and the need to remove VIBs at scale. Two scripts were created using get-esxcli and get-esxcli -V2!

Features of the scripts:

vCenter all host vib remove using get-esxcl V1 <–this was built on get-esxcli v1:

• It loops through hosts found in a given vCenter and removes the VIB(s) 1 by 1
• Allows the user to specify the VIB(s) by name and adds to a remove array
• Prompts for vCenter connection info

vCenter all host vib remove using get-esxcl -V2  <–This is the newer fully supported version built on get-esxcli -V2

• get-esxcli -V2 is the supported versions which allows “arguments” and a user to break down all of the variables
• This version prompts and asks for the following variable/arguments Dryrun, Maintenancemode, NoLiveInstall, and Force for force removal

Hopefully this can help you in mass removal of VIBs. Please reach out if you have questions!

vCloud Director: How to consolidate a VM with PowerCLI

With changes in the UI, features moving around or coming in later releases I was working on a consolidation script for VMs in vCloud Director.

Working with the Flash UI it was easy to see the “Chain Length” and consolidation in the VM menu:

 

Depending on versions of vCloud Director this may not be something exposed at the moment in the version being used. Even if it was exposed it might be nice to script the consolidation when needed.

The script will check if the VM that is to be consolidated is powered off and has a chain length longer than 1. If either of those checks fail it will return an error message. The script can be found in my gitlab blogpost files. Here is the output of it in action reporting my VM is powered on!

****************************************************************
Input the Org name: bblab
Input the vApp name: bbvapp
Input the VM name: bb

****************************************************************

****************************************************************

Could not consolidate VM bb within vApp bbvapp since it is PoweredOn

Please power off this VM and run script again
****************************************************************

Here is the expected return when it works!

****************************************************************
Input the Org name: bblab
Input the vApp name: bbvapp
Input the VM name: chain-test

****************************************************************
Processing VM: chain-test
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 100 / State: Success

 

Hopefully this is helpful!

vCloud Director: How to return default session(s) in powershell

While looking up something for a script idea in the works, google did not return any useful references for “How to return vCloud Director connected session in powershell/powercli” Thought I’d put this here as a reminder and resource in case others are looking for the same information!

$global:DefaultCIServers  # returns all sessions you are connected to, if scripting make sure you disconnect from sessions before scripting against an assumed session [0]

$global:DefaultCIServers[0]  # returns the first session you are connected to, does not appear to be alphabetical 

 $global:DefaultCIServers[0].Name  # can be used for variables or anything 

 $global:DefaultCIServers[0].ExtensionData  # tons of information if you dig in here 

Short, simple and hopefully helpful for others looking to return default sessions….and by “others” I also mean me in about 6 months when I forget how to do this again 😀  I had no clue how to do this but started tinkering around and found it.

vCloud Director: New/Set Catalog PowerShell Modules

A couple years ago I asked the #vExpert community for help to figure out how to create a new catalog within vCloud Director with powershell. There was no option to do so, and Jon Waite was amazing when he created the new-catalog module. The module works great to create a new catalog, but I wanted to take it another step to add in the publish and subscribe to external catalogs ability. I modified the module from Jon to add in these features. I am not 100% sure how to modify the module and check it into his original module so I did it in my blogpost files.

Giving full credit to Jon Waite for making this module. I have added the following to new-catalog.psm1:

• Ability to create a new catalog as published externally
• Script returns the published catalogs URL
• The above URL can be captured as a variable for use in other scripts (see below)

Set-cicatalog was created to change an existing catalog into an externally published catalog:

• Requires the catalog to already be created
• Script returns the published catalogs URL
• The above URL can be captured as a variable for use in other scripts (see below)

New-subscribedcatalog was created as I could not figure out how to put all of these features into 1 single module/function, which I might revisit:

• Ability to create a new catalog within vCD that is subscribed to an “external” feed
• Does minor error reporting if the catalog is created with a bad password and fails to sync
• Does minor status reporting and will return status of sync
• Allows variables to be passed in the script – meaning you can use the URL captured if the above modules 🙂

Hope this helps people in the future and please let me know if there are features I missed or something I can change to make it more user friendly!

vCloud Director: How to unsubscribe a subscribed catalog using PostgreSQL

Something that has perplexing for some time in vCloud Director is how to unsubscribed a catalog once it has been subscribed to an external feed. This is something that could be done to retain content without having to copy it to a local catalog. vCloud Director does not allow this to be done within the UI but it is a pretty quick change in the database

**Please note that these scripts are just for reference – use at your own risk** – Standard disclaimer 🙂 and take backups!!

Using either a SSH session or pgAdmin

The SSH session will be covered first(hopefully adding pgAdmin steps later):

Once a SSH session has been established with the PSQL DB machine, use the following commands to access the specific vCloud Director Database instance:

[root@bblab-vcddb ~]# su - postgres

Last login: Sat Jan 25 21:51:55 UTC 2020 on pts/0

-bash-4.2$ psql -d bblab-vcddb

psql (10.11)

Type "help" for help.

bblab-vcddb=#

Having accessed the vCloud Director database, it is time to track down the catalog in question. A safe way that I have found is to use the org_id to ensure you have tracked down the correct catalog within the given org that is being targeted. Copy the below and change the org name as needed:

select org_id from organization WHERE name = 'bblab-testorg1';

This will return the org_id from the database that is needed to ensure the proper catalog is selected and modified to no longer be subscribed.

select org_id from organization WHERE name = 'bblab-testorg1';

org_id

--------------------------------------

6a8b1172-7c13-4111-bb76-33a1b9471147

(1 row)

From there, take the org_id and catalog name in that needs to no longer be subscribed. Input those into the below statement and run!(please take a backup first – safety)

UPDATE catalog SET subscribed_to_ext_feeds = 'f' WHERE name = 'bblab-catalog1' AND org_id = '6a8b1172-7c13-4111-bb76-33a1b9471147';

Once the above has been ran, only 1 row will be updated. Checking the vCD UI, the catalog that was changed in the DB will no longer show subscribed and the items that had been synced will remain!

Hope this helps and I hope to post pgAdmin steps/screenshots in the near future!