vCloud Director: How to bypass SAML authentication for a tenant Org in the H5 UI

In the past tenants with Orgs in vCloud Director that were configured with SAML authentication were able to bypass the SAML authentication redirect by adding /login.jsp to the end of the Org URL. This would then allow a tenant to login with a local or LDAP account.

For example: https://bblab-vcd.com/cloud/org/testing-org/ would be changed to https://bblab-vcd.com/cloud/org/testing-org/login.jsp to allow local or LDAP authentication

Fast forward, there is no more FLEX/Flash UI and I could not find anything documented for the HTML5/H5 UI on how to do this. Trying to login to the Default organization URL(for example: https://bblab-vcd.com/tenant/testing-org/) would fail and could result in the following screen:

In the H5 UI, if the need arises to bypass the SAML authentication; /login could be added to the Default organization URL: https://bblab-vcd.com/tenant/testing-org/login which should redirect to something similar to this: https://bblab-vcd.com/login/login.jsp?service=tenant:testing-org&redirectTo=%2Ftenant%2Ftesting-org .Resulting in a much more pleasant page prompting for a User name/Password:

vCloud Director: HTML5 UI Login fails – “Failed to Start” error and fix

After upgrading a vCloud Director instance to 10.x, I’ve seen many logins fail with the following “Failed to Start” error message in the HTML5 UI only(**Note the FLEX UI is gone in 10.1 versions) This is covered in the following VMware KB as well: https://kb.vmware.com/s/article/75305

**Note this is found on vCD instances where the cells have been installed on linux, I’ve yet to test the appliance for the same**

Digging into this more, this error is caused by something called CORS filter not being updated with all values required. What is CORS and what values should be in CORS?

CORS stands for Cross-Origin Resource Sharing and is used by the H5 UI NOT Flex(flash) UI. The filter limits how each vCD endpoint can be accessed. In the configuration of the cells, if the CORS filter list doesn’t not contain the DNS, IP address, and shortname of each cell the login fails.

The workaround is a pretty quick and painless one. Run the following command on any cell in the vCD instance that is giving this error to retrieve the current CORS configuration:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n webapp.allowed.origins -l

The above will return something similar to this and in my case this is missing all 3 cell instances:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n webapp.allowed.origins -l ",10.1.1.1,http://10.1.1.3,http://10.1.1.2,http://10.1.1.1,https://10.1.1.2,https://10.1.1.1,https://10.1.1.3,https://vcd-bblab.com,http://vcd.bblab.com”

This list is comma separated, so simply adding in the missing values and making one small change to edit the values on the cell instead of listing them is all that is needed. To do that change the webapp.allowed.origins -l to a webapp.allowed.origins -v:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n webapp.allowed.origins -v "10.1.1.1,http://10.1.1.3,http://10.1.1.2,http://10.1.1.1,https://10.1.1.2,https://10.1.1.1,https://10.1.1.3,https://vcd-bblab.com,http://vcd.bblab.com,cell1.bblab.com,cell2.bblab.com,cell3.bblab.com,http://cell1.bblab.com,https://cell1.bblab.com,http://cell2.bblab.com,https://cell2.bblab.com,http://cell3.bblab.com,https://cell3.bblab.com”

Once the CORS values have been updated with the required information, logins will work once again without the need of restarting vCD cell services.

**Not supported, us at your own risk!!!**

These values are all stored in the vCloud Director database under the “config” table under “webapp.allowed.origins”

ESXi 7.0 Upgrade and Lifecycle Manager/VUM VIB dependencies

In my current position I’m working with upgrades at scale, and using  VUM/Lifecycle Manager (new to 7.0). Recently, I went to start the upgrade process on a number of hosts but ran into an issue pretty quick while trying to complete remediation. The exact error was this:


The upgrade has VIBs that are missing dependencies:

(List of 10 VIBs)

Remove the VIBs or use Image Builder to create a custom upgrade ISO image that contains the missing dependencies, and try to upgrade again.

LCM-error-blog

It should be easy to search a way to massively remove these VIBs and continue the upgrade… nope. Enter this blog post and the need to remove VIBs at scale. Two scripts were created using get-esxcli and get-esxcli -V2!

Features of the scripts:

vCenter all host vib remove using get-esxcl V1 <–this was built on get-esxcli v1:

  • It loops through hosts found in a given vCenter and removes the VIB(s) 1 by 1
  • Allows the user to specify the VIB(s) by name and adds to a remove array
  • Prompts for vCenter connection info

vCenter all host vib remove using get-esxcl -V2  <–This is the newer fully supported version built on get-esxcli -V2

  • get-esxcli -V2 is the supported versions which allows “arguments” and a user to break down all of the variables
  • This version prompts and asks for the following variable/arguments Dryrun, Maintenancemode, NoLiveInstall, and Force for force removal

Hopefully this can help you in mass removal of VIBs. Please reach out if you have questions!

vCloud Director: How to consolidate a VM with PowerCLI

With changes in the UI, features moving around or coming in later releases I was working on a consolidation script for VMs in vCloud Director.

Working with the Flash UI it was easy to see the “Chain Length” and consolidation in the VM menu:

chain-length1  chain-length2

Depending on versions of vCloud Director this may not be something exposed at the moment in the version being used. Even if it was exposed it might be nice to script the consolidation when needed.

The script will check if the VM that is to be consolidated is powered off and has a chain length longer than 1. If either of those checks fail it will return an error message. The script can be found in my gitlab blogpost files. Here is the output of it in action reporting my VM is powered on!


****************************************************************
Input the Org name: bblab
Input the vApp name: bbvapp
Input the VM name: bb

****************************************************************

****************************************************************

Could not consolidate VM bb within vApp bbvapp since it is PoweredOn

Please power off this VM and run script again
****************************************************************

Here is the expected return when it works!


****************************************************************
Input the Org name: bblab
Input the vApp name: bbvapp
Input the VM name: chain-test

****************************************************************
Processing VM: chain-test
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 0 / State: Running
chain-test - Percent: 100 / State: Success

 

Hopefully this is helpful!

vCloud Director: How to return default session(s) in powershell

While looking up something for a script idea in the works, google did not return any useful references for “How to return vCloud Director connected session in powershell/powercli” Thought I’d put this here as a reminder and resource in case others are looking for the same information!


$global:DefaultCIServers  # returns all sessions you are connected to, if scripting make sure you disconnect from sessions before scripting against an assumed session [0]

$global:DefaultCIServers[0]  # returns the first session you are connected to, does not appear to be alphabetical 
 $global:DefaultCIServers[0].Name  # can be used for variables or anything 
 $global:DefaultCIServers[0].ExtensionData  # tons of information if you dig in here 

Short, simple and hopefully helpful for others looking to return default sessions….and by “others” I also mean me in about 6 months when I forget how to do this again 😀  I had no clue how to do this but started tinkering around and found it.

vCloud Director: New/Set Catalog PowerShell Modules

A couple years ago I asked the #vExpert community for help to figure out how to create a new catalog within vCloud Director with powershell. There was no option to do so, and Jon Waite was amazing when he created the new-catalog module. The module works great to create a new catalog, but I wanted to take it another step to add in the publish and subscribe to external catalogs ability. I modified the module from Jon to add in these features. I am not 100% sure how to modify the module and check it into his original module so I did it in my blogpost files.

Giving full credit to Jon Waite for making this module. I have added the following to new-catalog.psm1:

  • Ability to create a new catalog as published externally
  • Script returns the published catalogs URL
  • The above URL can be captured as a variable for use in other scripts (see below)

Set-cicatalog was created to change an existing catalog into an externally published catalog:

  • Requires the catalog to already be created
  • Script returns the published catalogs URL
  • The above URL can be captured as a variable for use in other scripts (see below)

New-subscribedcatalog was created as I could not figure out how to put all of these features into 1 single module/function, which I might revisit:

  • Ability to create a new catalog within vCD that is subscribed to an “external” feed
  • Does minor error reporting if the catalog is created with a bad password and fails to sync
  • Does minor status reporting and will return status of sync
  • Allows variables to be passed in the script – meaning you can use the URL captured if the above modules 🙂

Hope this helps people in the future and please let me know if there are features I missed or something I can change to make it more user friendly!

vCloud Director: How to unsubscribe a subscribed catalog using PostgreSQL

Something that has perplexing for some time in vCloud Director is how to unsubscribed a catalog once it has been subscribed to an external feed. This is something that could be done to retain content without having to copy it to a local catalog. vCloud Director does not allow this to be done within the UI but it is a pretty quick change in the database

**Please note that these scripts are just for reference – use at your own risk** – Standard disclaimer 🙂 and take backups!!

Using either a SSH session or pgAdmin

The SSH session will be covered first(hopefully adding pgAdmin steps later):

Once a SSH session has been established with the PSQL DB machine, use the following commands to access the specific vCloud Director Database instance:


[root@bblab-vcddb ~]# su - postgres

Last login: Sat Jan 25 21:51:55 UTC 2020 on pts/0

-bash-4.2$ psql -d bblab-vcddb

psql (10.11)

Type "help" for help.

bblab-vcddb=#

Having accessed the vCloud Director database, it is time to track down the catalog in question. A safe way that I have found is to use the org_id to ensure you have tracked down the correct catalog within the given org that is being targeted. Copy the below and change the org name as needed:

select org_id from organization WHERE name = 'bblab-testorg1';

This will return the org_id from the database that is needed to ensure the proper catalog is selected and modified to no longer be subscribed.


select org_id from organization WHERE name = 'bblab-testorg1';

org_id

--------------------------------------

6a8b1172-7c13-4111-bb76-33a1b9471147

(1 row)

From there, take the org_id and catalog name in that needs to no longer be subscribed. Input those into the below statement and run!(please take a backup first – safety)


UPDATE catalog SET subscribed_to_ext_feeds = 'f' WHERE name = 'bblab-catalog1' AND org_id = '6a8b1172-7c13-4111-bb76-33a1b9471147';

Once the above has been ran, only 1 row will be updated. Checking the vCD UI, the catalog that was changed in the DB will no longer show subscribed and the items that had been synced will remain!

catalog1

Hope this helps and I hope to post pgAdmin steps/screenshots in the near future!

vCloud Director: Update PVDC supported hardware version via PowerCLI

This post servers as a reminder for me but also hopefully helpful to others that want to “explore” ExtensionData with powercli. This post shows how to declare a variable and navigate to the extension data for a Provider Virtual DataCenter within vCloud Director and adjust the Highest Supported Hardware Version.

**Please note that these scripts are just for reference – use at your own risk** – Standard disclaimer 🙂

Connect to a vCloud Director Instance:

Connect-CIServer bblab-vcd

Get a list of PVDCs within the vCD instance:


Get-ProviderVdc

Name Status Enabled CpuUsedGhz MemoryUsedGB StorageUsedGB
---- ------ ------- ---------- ------------ -------------
bblab-pvdc Ready True 8.43 (14.3%) 9.464 (13.0%)

Selecting one of the returned PVDCs above declare it as a variable:

 $hwversionchange = Get-ProviderVdc -name bblab-pvdc

Using the declared variable “pipe” “|” it to Format-List or FL to show more details(Note there is a HighestSupportedHardwareVersion here showing Unknown(possible bug):

 $hwversionchange | FL
Href : https://bblab-vcd/api/admin/extension/providervdc/uuid-here
StorageUsedGB :
StorageOverheadGB :
StorageTotalGB : 0
StorageAllocatedGB :
Status : Ready
MemoryUsedGB : 49.3623046875
MemoryOverheadGB : 1.73046875
MemoryTotalGB : 30.9296875
MemoryAllocatedGB : 14
Enabled : True
HighestSupportedHardwareVersion : Unknown
CpuUsedGHz : 17.164
CpuOverheadGHz : 0.384
CpuTotalGHz : 5.104
CpuAllocatedGHz : 4
ExtensionData : VMware.VimAutomation.Cloud.Views.VMWProviderVdc
StorageProfiles : {bblabcompute}
Description :
Name : bblab-pvdc

ExtensionData is one of the listed items above, adding a . after the variable will expose usable options. Type the PVDC variable and add .ExtensionData on the end to see a full list(Note as in above examples you can add any of these to the end of the variable.extensiondata to see more):

$hwversionchange.ExtensionData
HighestSupportedHardwareVersion : vmx-13
ResourcePoolRefs : {VMware.VimAutomation.Cloud.Views.VimObjectRef1}
VimServer : {bblab-vc1}
DataStoreRefs : {VMware.VimAutomation.Cloud.Views.VimObjectRef1, VMware.VimAutomation.Cloud.Views.VimObjectRef1, VMware.VimAutomation.Cloud.Views.VimObjectRef1,
VMware.VimAutomation.Cloud.Views.VimObjectRef1...}
HostReferences : VMware.VimAutomation.Cloud.Views.VMWHostReferences
NsxTManagerReference :
Status : 1
StorageCapacity :
ComputeCapacity : VMware.VimAutomation.Cloud.Views.RootComputeCapacity
IsEnabled : True
AvailableNetworks : VMware.VimAutomation.Cloud.Views.AvailableNetworks
Vdcs :
Capabilities :
StorageProfiles : VMware.VimAutomation.Cloud.Views.ProviderVdcStorageProfiles
NetworkPoolReferences : VMware.VimAutomation.Cloud.Views.NetworkPoolReferences
Tasks :
Description :
OperationKey :
Client : VMware.VimAutomation.Cloud.Views.CloudClient
Type : application/vnd.vmware.admin.vmwprovidervdc+xml
Link : {, , , ...}
AnyAttr :
VCloudExtension

Simplifying things the HighestSupportedHardwareVersion  has been added to the command:

$hwversionchange.ExtensionData.HighestSupportedHardwareVersion

vmx-13

Keeping the format from above I know I can increase the value to vmx-14 based on the vCloud Director Admin Guide. Set the value with the below command:

$hwversionchange.ExtensionData.HighestSupportedHardwareVersion = 'vmx-14'

The value needs to be “committed” by using the “UpdateServerData()” command which will push the updated value to the vCD instance(Note – make sure you have a backup of your vCD Database in case something goes wrong):

 $hwversionchange.ExtensionData.UpdateServerData()

Once the above command has finished it will return all of the ExtensionData including the newly changed first line 

HighestSupportedHardwareVersion : vmx-14

There is plenty to explore with ExtensionData in vSphere and vCloud Director – explore safely……… with backups 😉

vCloud Availability for Cloud-to-Cloud DR aka vCAV Install and Config part 3

Building off of Part 1 & Part 2 which covers “what is vCAV and how to use vCAV”, I’ll detail in this post, the configurations of policies, protect VMs/vApps, do a migration and cleanup migrated VMs/vApps that no longer need to be protected.

*Once again disclaimer – I’m writing this for the 3.0 Beta version and features. The process outlined below may change – 3.0.1 is GA as of the 23rd of May 2019 release notes found here

Before protecting a VM or vApp to be migrated to a new cloud site the policies will need to be configured on each site. Policies can be created for use by a single Org or can be assigned any combination of Orgs per policy. Each policy can be configured to allow outgoing or incoming replications, to limit the number of replications and snapshots per replications, and the minimum allowed RPO. A fantastic explanation for RPO can be found here. The RPO value can be changed per workload protection but this setting is for the minimum allowed value.

If a new policy is not created, the following error will appear when attempting to protect a new workload:vcav-protect-vapp3-policyerror

To create a new policy, navigate to Policies on the left hand navigate pane and click NEW(The default policy can be edited to allow replications and by default all discovered orgs are attached to this policy):vcav-protect-vapp4-policy-new.PNG

The New Policy window will pop up and each name and setting can be adjusted for each policy created:

vcav-protect-vapp5-policy-new.png

Once created, click the radio button next to the policy to assign a new org or orgs and click the ASSIGN button above the polices:

vcav-protect-vapp6-assign-policy-new

Once the policies have been assigned to the orgs required, vApps can now be protected. Navigate to the left hand side and click from Cloud or to Cloud depending on the direction of the migration:

protect-vapp1

For this example to Cloud was selected under Outgoing Replications – the on the top menu click NEW to begin protecting a vApp.

The New Outgoing Replication menu appears and allows to sort by the vApp/VM or OrgVDC(VDC aka OVDC) to find the items that need to be protected:

***Note the yellow warning banner notes the VM(s) are powered off and a seed will not be created on the destination side unless done manually or the VM(s) becomes powered on***protect-vapp2

After clicking NEXT, the Target Site needs to be selected and authenticated to along with the Target VDC(OVDC). Once these have been configured the next menu will show the Protection Settings:. Below for each migration there are a number of values that can be configured based on the policy settings created above. RPO, Storage policies, does the VM support quiesceing, and compress replication traffic. Each of these is configurable for each protection.vcav-replication-test2

The next menu is Scheduling, which is a fantastic add in 3.x versions! This allow the synchronization of the selected workloads to begin as a specified time or done immediately:

protect-vapp3

Lastly a summary page Ready To Complete appears detailing the protection settings for the selected workloads. Clicking on FINISH will complete the wizard and file the job within the vCAV appliances!

What happens when vApps/VMs are migrated?

When a vApp or VM is migrated with vCAV, all of the data is synced over to the destination side, the source VM is powered off gracefully and the destination becomes ready in vCD(depending on the power state selected during migration, the power state may be on/off).

What happens when vApps/VMs are “failed-over”?

This assume the source side is down and is more of a disaster recovery option. The Source side is not touched in this case, and the destination side will have the last synchronized data based on the selected RPO.

*Please note the below feature was available in 1.5.

How to report on usage:

Login to the vCloud Availability vApp Replication Manager as root with the following command

c4 loginroot C4-Root-Password-Here

Generate the vCloud Availability for Cloud-to-Cloud DR usage report(report_summary and report_details can be changed to any report name)

usage-report --output /tmp/report_summary.tsv --details /tmp/report_details.tsv

Download the vCloud Availability for Cloud-to-Cloud DR usage report locally.

scp /tmp/report_summary.tsv /tmp/report_details.tsv user@your-host:/download-target-location

(Optional) Remove the generated reports from the vCloud Availability vApp Replication Manager appliance.

rm /tmp/report_summary.tsv /tmp/report_details.tsv
Notes for the above usage report can be found here:

https://docs.vmware.com/en/VMware-vCloud-Availability-for-Cloud-to-Cloud-DR/1.5/com.vmware.c2c.install.config.upgrade.doc/GUID-F36C3CDD-2477-48DE-9883-333D1828A969.html

A HUGE thanks to the vCAV product team that I’ve had the pleasure of working closely with in my current role. They are striving to get to increase scale and get new features into the product quickly! This product just keeps getting better with every release.

Please let me know if you have feedback or question – I’ll do my best to answer them!

 

 

Gifts for Tech People

My wife tells me I’m difficult to shop for.  I believe it, as I have heard this from a few different sources.  As a techy person, tech gifts aren’t something I often receive, and to be fair, when they are given, I probably could have found it better and less expensive!  I have found a few things that are less common that other tech people might love.  If nothing else, I hope I will ignite the spark of a new idea for your techy loved one… and if you want to send any of these my way, well, that’s fine, too.

Key knife – infinitely useful and stays on your key chain, blending in with other keys.  The main purpose I use this for is opening packages, both from Amazon and on birthdays/holidays.  Honestly, I use my key knife daily. Most recently, we used it to free a distressed and trapped garter snake from a cheap plastic fence in our local botanical garden/sculpture park!  It’s always at the ready and handy.  (The snake is fine now, by the way!)

Isis puzzle ball – Looks simple enough but provides hours of puzzle solving enjoyment. Once you take this on you will want the next in the series to solve.

Nerdy Tie Dyes – When my wife found out I was making this blog post, she insisted we include our favorite tie dye shop!  This girl does the best tie dyes that are so bright and cheerful!  She can do swirls and all that jazz, of course.  But her custom designs are where she shines! We have gotten guitars, Darth Vader, rainbows, hearts, turkeys, a peeps bunny, and so much more swirled into shirts.  When we had our last baby, we got several newborn onesies that were beautiful. We didn’t find out the sex, so we loved having not only gender neutral clothes at first, but truly beautiful clothes perfect for photo ops (and hiding barf). She is great to work with and we have always been pleased with the quality and craftsmanship of her dyes.  We also got a special offer for anyone who wants to order from her through this blog! Enter the coupon code NERDYDYES for 15% off of anything in the shop and free shipping (to U.S. customers)!

Seagull Mobile – makes it onto my list as the most inconspicuous and surprising item that I’m sure no one will have.  I have this hanging from the ceiling over my chair at my desk. I get so deep into work that I forget to take deep breaths or blink or whatever.  All I do is tilt my head back and try to move the seagulls with my breath.  It’s really, really simple, but really effective and calming.  Any mobile or thing hanging from a string would work just as well.

Darn Tough socks – I love these socks! Made in the U.S.A., they have a lifetime gaurantee and are so comfortable! They’re just made better than other socks.  They allow your feet to breathe, and do such a good job of wicking moisture and smell that I’ve worn them for days… Amazing for traveling, growing boys and stinky feet.

Theraband bands/mini tramp/exercise ball- I’m all about small breaks that make your work day better by clearing your mind, even if the break is only for a minute or two.  It’s not enough time to sweat, but an active quick break for mental health!

Lodge Cast Iron griddle – I love cast iron!  This griddle is the creme de la creme when it comes to cast iron.  It has a grill side and a flat side, which is the one we almost always use.  It cleans up so shiny and has a commercial kitchen feel to it; you get more space to cook. Everything cooks so beautifully and quick, even when compared to the rest of our cast iron.  We use it for potatoes, eggs, paninis, veggies, pancakes, English muffins, burgers, etc.

GIPF series of games/7 wonders duel – These are some of the best games I have ever played. They are all two person games.  The strategy is different in all of them, and I love the adjustment of acclimating to the strategy needed to win.  They’re all super simple yet complex, and MOST people don’t have them (nor have ever heard of them)! Each one of us has a different favorite! 7 Wonders Duel was given to me by an great friend and we’ve been playing at least once a week since.

Car trunk organizer – This has been a much needed addition for years.  It was one of those purchases that after I had them, we both had this, “where have these been our whole lives?” kind of response.  I have two: one for the staples that stay in the car (jumpers, reusable bags, etc.) and one that is used solely for groceries (so we don’t forget anything in the trunk). They fold up nicely and barely take up any space.

Rice bag heat packs – I get these at local artist markets, but you could get them at Etsy or anywhere.  They have been amazing for any mild aches, pains, sore muscles, cold nights, etc. There are so many uses for them.

Garden seeds – Request a catalog from this company and it’ll have you dreaming of spring with ambitions of growing your own beautiful harvest this year!

Pump-its – They cut anything you can think of that fits the diameter the tool says it can take. At the time we got them my 6 year old had no trouble cutting branches with these!

Merrell sandals – Hands down the most comfortable shoe!  I use these for long periods of time in the kitchen (when my wife has me slaving away without breaks for weeks) and they always keep my feet nice and happy.

Sherlock holmes/mystery book – a good mystery book with great vocabulary and British humor is always a great idea.

Foraging book – getting away from the screen is really important to me, and getting out into nature is fun and relaxing- but we wanted more!  Our questions have turned into knowledge and we have had some amazing trail side snacks and meals from food we have found!  My favorite is a cranberry and autumn olive sauce at Thanksgiving! But then there’s also wild garlic, morels, ramps, wild black raspberries, sassafras, mint, wild cherries, dandelion, apples, onions, grapes, wild mustard greens, rosehips, and so much more.  It’s awakening and inspiring when you step outside and a plant becomes a familiar culinary friend. The books listed are more specific to the Midwest U.S., so maybe find one indigenous to your area! If nothing else, this author is a great writer and he makes for some lovely reading.

If you’ve made it this far, you’re super desperate for a gift…. when everything else fails, gift cards still work 🙂