Digital Edition

SYS-CON.TV
Stupid Mistake Cripples VMware Servers
A Time Bomb Left Over from the Beta of VMware's Two-Week-Old Update to ESX 3.5 and ESXi 3.5 Caused Product Licenses to Expire

A time bomb reportedly left over from the beta of VMware’s two-week-old update to ESX 3.5 and ESXi 3.5 (Update 2) caused product licenses to expire yesterday.

It also happened with a patch to ESX 3.5 or ESXi 3.5 Update 2.

The experience affected a whole lot of people, apparently enough to crash VMware’s knowledge base.

VMware put out a freebie ESXi hypervisor on July 28 as a way of coping with Microsoft and its free Windows Server 2008-embedded Hyper-V.

As a result of the oversight, VMware virtual machines that were turned off couldn’t be turned back on; virtual machines that were suspended couldn’t get out of suspend mode; and virtual machines couldn’t be migrated with VMotion.

All in all it wasn’t a good day for VMware – whose new CEO Paul Maritz found himself posting an open letter of apology to the VMware web site shortly before midnight New York time last night.

It also wasn’t good for the company’s QA people who left the cursed code in the releases. They are now threatened with a long session in the virtual stocks.

“We failed in two areas,” Maritz’s letter says, “not disabling the code in the final release of Update 2 and not catching it in our quality assurance process.”

“This incident.” he said, “has prompted a thorough self-examination of how we create and deliver products to our customers. We have kicked off a comprehensive, in-depth review of our QA and release processes, and will quickly make the needed changes.”

VMware had to deny that the issue had anything to do with a security problem.

Maritz posted his letter after the company issued a temporary so-called “express patch” – two of them actually, one for ESX 3.5, one for ESXi Server 3.5 – and he promised “within the next 24 hours, we also expect to issue a full replacement for Update 2.”

VMware said simply resetting the host time to a date prior to August 12, 2008 had “a number of very serious side effects that could impact product environments”; anyway that patchwork workaround wouldn’t meet governance requirements.

The bug cost users at least 20 hours of downtime, not to mention the dye to cover their newly gray hairs and the effort to put everything back together again.

It also happened just as a lot of servers were being rebooted because it was Microsoft’s Patch Tuesday.

The incident also made them think about deploying multiple hypervisors, not something VMware wants to hear.

About Maureen O'Gara
Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025. Twitter: @MaureenOGara

In order to post a comment you need to be registered and logged in.

Register | Sign-in

Reader Feedback: Page 1 of 1



ADS BY GOOGLE
Subscribe to the World's Most Powerful Newsletters

ADS BY GOOGLE

The standardization of container runtimes and images has sparked the creation of an almost overwhelm...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018,...
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point wh...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22...
Dynatrace is an application performance management software company with products for the informatio...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, ...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 1...
A valuable conference experience generates new contacts, sales leads, potential strategic partners a...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As au...
DXWorldEXPO LLC announced today that ICOHOLDER named "Media Sponsor" of Miami Blockchain Event by Fi...
SYS-CON Events announced today that IoT Global Network has been named “Media Sponsor” of SYS-CON's @...
"We do one of the best file systems in the world. We learned how to deal with Big Data many years ag...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news an...
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held Novemb...
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing w...
Enterprise architects are increasingly adopting multi-cloud strategies as they seek to utilize exist...
Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear...
In this Women in Technology Power Panel at 15th Cloud Expo, moderated by Anne Plese, Senior Consulta...
Enterprises are striving to become digital businesses for differentiated innovation and customer-cen...
DXWorldEXPO LLC announced today that Telecom Reseller has been named "Media Sponsor" of CloudEXPO | ...