Ubuntu 18.04 End of Standard Support - What it means for Google Cloud VMware Engine Environments

Recently, you may have received an e-mail notice from the Google Cloud team about the “End of Standard Support” for Ubuntu 18.04 LTS (Bionic Beaver) on May 31, 2023. According to an Ubuntu blog post, now is the time to take action because machines running that version will not receive any security updates after that date. Whether you have Ubuntu instances in Compute Engine for management or Ubuntu virtual machines in your SDDC, this post will give some tips and discuss some snags I encountered while upgrading.

Check your OpenVPN Point-to-Site VPN Appliance

When I first read the e-mail advisory, I didn’t recall ever using Ubuntu 18.04 LTS in my Compute Engine environment, so I had to investigate. It turns out that the OpenVPN Point-to-Site appliance I deployed last November was running version 18.04. To remedy the situation, I decided to deploy a second OpenVPN appliance so that I could perform a side-by-side migration. (I figured that would help avoid any unexpected problems or incompatibilities with manually upgrading the appliance operating system.)

Figure 1: The latest Open VPN Access Server version 2.11.3 that uses Ubuntu 22.04 LTS

Since recording the demo video for Google Cloud VMware Engine Point-to-Site VPN Configuration (which used OpenVPN Access server 2.8.5), there was only one minor difference in the deployment process with 2.11.3. The step in the demo video where you needed to change the openvpn temporary password from the SSH login window is no longer necessary, and everything can now be done from the admin console. Otherwise, the remaining configuration steps were identical. Once I deployed the appliance, set a static IP, transferred my host settings, and updated my client settings, I was done in under 20 minutes.

Note: If you need help through this process, the details screen for the OpenVPN appliance within the Deployment Manager has an updated link to the GCP Quick Start Guide.

Don't forget about the Ubuntu virtual machines you have already migrated

Depending on where you are in your cloud journey, you may have already discovered the ease with which you can migrate your workloads to Google Cloud VMware Engine using VMware HCX. It is important to remember that although you can migrate those enterprise apps from your on-prem data center to the cloud easily and with near-zero modifications, they might still need to be upgraded from 18.04 too. Although there are some tools or scripts that can be run to scan the environment for out-of-date operating systems, the vSphere client can also help you quickly find Ubuntu VMs that could be potential upgrade candidates.

Timesaver tip: As long as the virtual machines have the correct operating system defined, you can use the Global Inventory Lists within your vCenter console. From the vSphere client: Menu > Global Inventory Lists > Virtual Machines > Click a Column arrow ∨ > Show/Hide Columns > Guest OS

Figure 2: Using the Global Inventory Lists to identify Ubuntu virtual machines

Upgrading to Ubuntu 22.04 LTS – Check your SSH Keys beforehand

Once my OpenVPN appliance was replaced, I decided to upgrade my management machine that I have running in GCP. This machine is what I use for various management tasks like gcloud commands, ping testing, etc. Although it was already running Ubuntu 20.04 LTS, I decided this would be an excellent time to upgrade to 22.04 LTS.

However, I hadn’t anticipated that the “ssh-rsa” public key signature algorithm was deprecated in the SSH daemon included in 22.04. To be clear, the rsa key type is still available for use with RSA signature variants such as "rsa-sha2-256" and "rsa-sha2-512"; it is the SHA-1 signature used with "ssh-rsa" that is disabled.

This meant that I could no longer use my old private/public keys and couldn’t access my administrative virtual machine.

When I tried to log in, I was greeted with a message saying:

Server refused our key
No supported authentication methods available (server sent: publickey)

Thankfully, I could connect via SSH to the instance from Compute Engine console to implement a suggested workaround from the Ubuntu user community.

Figure 3: Workaround - Updating the /etc/ssh/sshd_config file to re-enable the ssh-rsa key scheme temporarily

Although the ideal solution would have been to swap out keys beforehand proactively, the workaround allowed me to get back in to resolve the issue. From the Compute Engine SSH connect window :

$ sudo nano /etc/ssh/sshd_config

#Add the following lines

HostKeyAlgorithms +ssh-rsa
PubkeyAcceptedKeyTypes +ssh-rsa

#Restart the sshd daemon

$ systemctl restart sshd

Note: Once your keys have been replaced on the hosts, you can delete those two lines and restart sshd.

Ultimately, I decided to switch to the ed25519 scheme and replace my keys everywhere, which meant more work but hopefully fewer headaches in the future. For more information about generating keys, the ssh-keygen manual provides a comprehensive guide on the command syntax. In addition, the Google Cloud documentation provides examples of how to Create SSH keys for different operating systems. The best key type for your organization is beyond the scope of this post, but key type comparisons can be searched for online to get more in-depth information.

Timesaver tip: Add your new key in the Compute Engine > Settings > Metadata > SSH Keys screen. This will save you from manually adding it to the authorized keys on each GCP instance.

Takeaway

Hopefully, these tips will help you avoid unforeseen issues through Ubuntu upgrades within your Google Cloud and VMware Engine environments. For more information about Google Cloud VMware Engine, please visit VMware Cloud Tech Zone.