Digital Edition

SYS-CON.TV
Stupid Mistake Cripples VMware Servers
A Time Bomb Left Over from the Beta of VMware's Two-Week-Old Update to ESX 3.5 and ESXi 3.5 Caused Product Licenses to Expire

A time bomb reportedly left over from the beta of VMware’s two-week-old update to ESX 3.5 and ESXi 3.5 (Update 2) caused product licenses to expire yesterday.

It also happened with a patch to ESX 3.5 or ESXi 3.5 Update 2.

The experience affected a whole lot of people, apparently enough to crash VMware’s knowledge base.

VMware put out a freebie ESXi hypervisor on July 28 as a way of coping with Microsoft and its free Windows Server 2008-embedded Hyper-V.

As a result of the oversight, VMware virtual machines that were turned off couldn’t be turned back on; virtual machines that were suspended couldn’t get out of suspend mode; and virtual machines couldn’t be migrated with VMotion.

All in all it wasn’t a good day for VMware – whose new CEO Paul Maritz found himself posting an open letter of apology to the VMware web site shortly before midnight New York time last night.

It also wasn’t good for the company’s QA people who left the cursed code in the releases. They are now threatened with a long session in the virtual stocks.

“We failed in two areas,” Maritz’s letter says, “not disabling the code in the final release of Update 2 and not catching it in our quality assurance process.”

“This incident.” he said, “has prompted a thorough self-examination of how we create and deliver products to our customers. We have kicked off a comprehensive, in-depth review of our QA and release processes, and will quickly make the needed changes.”

VMware had to deny that the issue had anything to do with a security problem.

Maritz posted his letter after the company issued a temporary so-called “express patch” – two of them actually, one for ESX 3.5, one for ESXi Server 3.5 – and he promised “within the next 24 hours, we also expect to issue a full replacement for Update 2.”

VMware said simply resetting the host time to a date prior to August 12, 2008 had “a number of very serious side effects that could impact product environments”; anyway that patchwork workaround wouldn’t meet governance requirements.

The bug cost users at least 20 hours of downtime, not to mention the dye to cover their newly gray hairs and the effort to put everything back together again.

It also happened just as a lot of servers were being rebooted because it was Microsoft’s Patch Tuesday.

The incident also made them think about deploying multiple hypervisors, not something VMware wants to hear.

About Maureen O'Gara
Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025. Twitter: @MaureenOGara

In order to post a comment you need to be registered and logged in.

Register | Sign-in

Reader Feedback: Page 1 of 1



ADS BY GOOGLE
Subscribe to the World's Most Powerful Newsletters

ADS BY GOOGLE

At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with exp...
When a company wants to develop an application, it must worry about many aspects: selecting the infr...
Financial enterprises in New York City, London, Singapore, and other world financial capitals are em...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with exp...
Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the U...
DevOps has long focused on reinventing the SDLC (e.g. with CI/CD, ARA, pipeline automation etc.), wh...
Most modern computer languages embed a lot of metadata in their application. We show how this goldmi...
Cloud Storage 2.0 has brought many innovations, including the availability of cloud storage services...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with exp...
Every organization is facing their own Digital Transformation as they attempt to stay ahead of the c...
Intel is an American multinational corporation and technology company headquartered in Santa Clara, ...
Data center, on-premise, public-cloud, private-cloud, multi-cloud, hybrid-cloud, IoT, AI, edge, SaaS...
Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures....
Today's workforce is trading their cubicles and corporate desktops in favor of an any-location, any-...
Artifex Software began 25-years ago with Ghostscript, a page description language (PDL) interpreter ...
ShieldX's CEO and Founder, Ratinder Ahuja, believes that traditional security solutions are not desi...
Is your enterprise growing the right skills to fight the digital transformation (DX) battles? With 6...
In an age of borderless networks, security for the cloud and security for the corporate network can ...
Now is the time for a truly global DX event, to bring together the leading minds from the technology...
Moving to Azure is the path to digital transformation, but not every journey is effective. Organizat...