Tech

VMC – Moving from VPN to DX and reconfiguring HCX Service Mesh

by Frances Wong

Much of my job entails post sales support, design, re-design, deployment, and rearchitecting of the VMC solution so I have to answer many questions related to the growth and expansion of customer uses of VMC.

Recently, a customer needed some guidance when it came to taking their use of VMware Cloud on AWS to the next level.

Like many customers that start out, they had a 3 node VMC SDDC stood up connected via a VPN tunnel over the public Internet. They also had HCX deployed on premise with multiple stretched networks and two HCX Service Meshes.

The next phase of their transition was to install two Direct Connects and reconfigure HCX from the VPN Tunnels to the Direct Connects.

Much of VMware’s documentation covers installing a Direct Connect but does not follow up on the HCX reconfiguration. I had to cross both sets of documentation and piece them together to provide the client a end to end process.

Hopefully, some of you can use this.

VMC Day 2 Operations – Reconfiguring HCX Service Mesh after moving VMC management connection from a VPN to a Direct Connect.

Task:  Customer operating HCX over an HCX VPN connection over the Internet. Customer establishes new Direct Connect to existing SDDC. Customer needs to move HCX Service Mesh to operate over Direct Connect from the current HCX VPN Connection. Customer also has VPN established between on premise and VMC and will be moving to a Direct Connect connection to VMC only.

Steps:

  1. Un-deploy existing HCX Service Mesh(s). #Not required as this has no impact on Management VPN. However, my prior experience as a admin/architect tells me that if I have to have an outage, the cleanest way to perform maintenance is to remove the active piece during the infrastructure work in order to not have to troubleshoot scenarios where things look like they are ok but are not ok under the hood because of the maintenance work.
  2. Remove VPN connection in VMC. #Not required. This is based on this client knowing that they were going to drop the VPN altogether. If you plan on keeping the VPN, it’s fine to keep it up. Make sure when the Direct Connect is up, you know how to verify connectivity for both connections.
  3. Verify Direct Connect is connected to VMC SDDC. Here are the steps to connect an SDDC to a DX connection.
  4. Check advertised routes are correct from Direct Connect. (including vsphere management, HCX management, vCenter)
  5. Reconfigure/verify/create new management firewall rules in VMC SDDC that referred to the prior VPN connection to the Direct Connect.
  6. Ping cloud VMC SDDC vCenter from one of the on premise networks that is being advertised over the Direct Connect to verify the on premise vSphere management can communicate with VMC Management appliances and network.
  7. HCX – if you kept the Service Meshes up, ensure there are no live migrations and/or any new configurations in progress. Change the HCX fully qualified domain name (FQDN) from VMC cloud service console->Settings. 
    1. Click the down Chevron and click Edit 
    2. Select the Private IP: X.X.X.X from the Resolution Address drop down menu and click Save. 
  8. Verify the resolution of the HCX FQDN is returning the internal IP address. 
    1. Login to VMC Cloud service console (https://vmc.vmware.com) 
    2. Get the cloudadmin account info from the VMC cloud services console->Settings 
    3. Get the HCX fully qualified domain name (FQDN) from VMC cloud service console->Settings 
  9. Verify HCX Connector & HCX Cloud pairing is healthy or create the necessary site pairing.
  10. Edit Direct Connect Network Profile in HCX Cloud in VMC.
    1. Login to HCX cloud instance with thecloudadmin@vmc.local account 
    2. Edit the DirectConnectNetwork1 network profile 
    3. In the cloud HCX manager, click Infrastructure->Interconnect->Network Profiles 
    4. Click Edit on the directConnectNetwork1 network profile 
    1. Add the IP range, prefix length, and gateway to the network profile and click Update. (Make sure no overlapping IP ranges in cloud or on premise!)
    2. After a couple of minutes, validate that the network segment is advertised over the DX in VMC SDDC Console ->Networking & Security->System->Direct Connect.  The subnet should be visible under Advertised BGP Routes. 
    3. Verify that your on premise devices are allowing the new HCX IP Range to communicate with the vSphere Management network on premise.
  11. Redeploy HCX Service Mesh(s) from on premise HCX Connector Manager
    1. During the service mesh build in step 3, select the directConnectNetwork1 network profile for the Destination Site Uplink Network Profile. 
  12. Complete HCX Service Mesh build. 
  13. Reconfigure/verify/create new compute firewall rules in VMC SDDC to referred to the prior VPN connection to the Direct Connect .

I apologize if these notes are a little raw. I know there are many small things that I didn’t expand or explain here. Your installation may vary quite a bit so modify as needed. These steps are provided as a roadmap for you to make sure you encompass all the various elements to make sure the move is successful.