Upgrading VCSA 6.7 to VCSA 8 fail! – “Encountered an internal error”
I was upgrading a customer’s vCenter Server from version 6.7 to 8. While the upgrade is supported, I encountered warnings (but no errors) during the pre-checks in stage 2. I ignored these warnings and proceeded, which resulted in an upgrade error.
Encountered an internal error. Traceback (most recent call last): File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 1175, in main vmidentityFB.boot() File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 273, in boot self.configureSTS(self.__stsRetryCount, self.__stsRetryInterval) File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 797, in configureSTS self.startSTSService() File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 757, in startSTSService returnCode = self.startService(self.__sts_service_name) File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 78, in startService update_services_runstate("start", None, False, False, svc_names=[svc_name]) File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 1122, in update_services_runstate _update_services_runstate_svclist('start', svc_nodenames, and....It is similar to this KB but I have checked DNS again and it was not a DNS problem. I did deploy the vCenter appliance once again. This time, at stage two, I took a closer look and noticed the warnings I had skipped previously.
- SSL Trust certificate does not match the current MACHINE_SSL_CERT for one of the service registrations
- A legacy service registrations for SSO service has been found to still use Port 7444
A legacy service is a service that has been migrated from previous versions of vCenter.
You will find here more information about these warnings, https://kb.vmware.com/s/article/79741, resolving these before upgrading is recommended to avoid potential failures.
The solution is easy! using a Doctor! yeah! learn more about lsdoctor tool here. Now let’s go to the solution:
1- Make sure you have a backup from your vCenter.
2- Make a snapshot from vCenter
3- Download the zip from this link.
4- Unzip the downloaded file in a folder like tmp and make sure you are in lsdoctor directory
unzip lsdoctor-240201.zip5- To correct the SSL trust mismatch issues in the lookup service, run the following command:
python lsdoctor.py -t
6- To clean up any stale configurations left over from the previous upgrade and version, run the following command:
7- Stop and start the vCenter services.
service-control --stop --all
service-control --start --all8- Continue to your upgrade! That worked perfectly for me!
Hope this is informative for you!
Hi Yahya,
Thanks for this useful article.
I suspect I am experiencing trust issues because cold migrates that were working OK are now falling before starting. There is an unknown ssl error at the nfcmanager level in the vpxa log.
I have an issue with LS Doctor though. I have posted this in the VMWare Community forum but thought you might like to know about it and have some suggestions about how to fix it too.
Error is
File “/lsdoctor-240201/lib/lsdoctor_defaults.py” line 2, in import importlib.util
ImportError: No module named util
Investigating the content of lsdoctor_defaults.py shows it is important to the functioning of trustfix as it seems to import the existing VCSA system and os values into a decorator called @contextmanager
def get_vmidentity_classpath():
with suppress_stdout():
spec = importlib.util.spec_from_file_location(“VMwareIdentityFirstBoot”,
“/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py”)
vmidfb = importlib.util.module_from_spec(spec)
sys.modules[“VMwareIdentityFirstBoot”] = vmidfb
spec.loader.exec_module(vmidfb)
cp = vmidfb.VMwareIdentityFirstBoot()
class_path = cp._VMwareIdentityFirstBoot__classpath
return class_path
Because this looks so fundamental to the correct operation of trustfix I am reluctant to execute the rest of the script and exited by not confirming the existence of an offline snapshot.