Digital Edition

SYS-CON.TV
Stupid Mistake Cripples VMware Servers
A Time Bomb Left Over from the Beta of VMware's Two-Week-Old Update to ESX 3.5 and ESXi 3.5 Caused Product Licenses to Expire

A time bomb reportedly left over from the beta of VMware’s two-week-old update to ESX 3.5 and ESXi 3.5 (Update 2) caused product licenses to expire yesterday.

It also happened with a patch to ESX 3.5 or ESXi 3.5 Update 2.

The experience affected a whole lot of people, apparently enough to crash VMware’s knowledge base.

VMware put out a freebie ESXi hypervisor on July 28 as a way of coping with Microsoft and its free Windows Server 2008-embedded Hyper-V.

As a result of the oversight, VMware virtual machines that were turned off couldn’t be turned back on; virtual machines that were suspended couldn’t get out of suspend mode; and virtual machines couldn’t be migrated with VMotion.

All in all it wasn’t a good day for VMware – whose new CEO Paul Maritz found himself posting an open letter of apology to the VMware web site shortly before midnight New York time last night.

It also wasn’t good for the company’s QA people who left the cursed code in the releases. They are now threatened with a long session in the virtual stocks.

“We failed in two areas,” Maritz’s letter says, “not disabling the code in the final release of Update 2 and not catching it in our quality assurance process.”

“This incident.” he said, “has prompted a thorough self-examination of how we create and deliver products to our customers. We have kicked off a comprehensive, in-depth review of our QA and release processes, and will quickly make the needed changes.”

VMware had to deny that the issue had anything to do with a security problem.

Maritz posted his letter after the company issued a temporary so-called “express patch” – two of them actually, one for ESX 3.5, one for ESXi Server 3.5 – and he promised “within the next 24 hours, we also expect to issue a full replacement for Update 2.”

VMware said simply resetting the host time to a date prior to August 12, 2008 had “a number of very serious side effects that could impact product environments”; anyway that patchwork workaround wouldn’t meet governance requirements.

The bug cost users at least 20 hours of downtime, not to mention the dye to cover their newly gray hairs and the effort to put everything back together again.

It also happened just as a lot of servers were being rebooted because it was Microsoft’s Patch Tuesday.

The incident also made them think about deploying multiple hypervisors, not something VMware wants to hear.

About Maureen O'Gara
Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025. Twitter: @MaureenOGara

In order to post a comment you need to be registered and logged in.

Register | Sign-in

Reader Feedback: Page 1 of 1



ADS BY GOOGLE
Subscribe to the World's Most Powerful Newsletters

ADS BY GOOGLE

SUSE is a German-based, multinational, open-source software company that develops and sells Linux pr...
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism...
NanoVMs is the only production ready unikernel infrastructure solution on the market today. Unikerne...
Big Switch's mission is to disrupt the status quo of networking with order of magnitude improvements...
Yottabyte is a software-defined data center (SDDC) company headquartered in Bloomfield Township, Oak...
Dynatrace is an application performance management software company with products for the informatio...
Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in developm...
All in Mobile is a mobile app agency that helps enterprise companies and next generation startups bu...
Every organization is facing their own Digital Transformation as they attempt to stay ahead of the c...
Blockchain is a new buzzword that promises to revolutionize the way we manage data. If the data is s...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the m...
Serveless Architectures brings the ability to independently scale, deploy and heal based on workload...
I spend a lot of time helping organizations to “think like a data scientist.” My book “Big Data MBA:...
The standardization of container runtimes and images has sparked the creation of an almost overwhelm...
Whenever a new technology hits the high points of hype, everyone starts talking about it like it wil...
Wasabi is the hot cloud storage company delivering low-cost, fast, and reliable cloud storage. Wasab...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitori...
David Friend is the co-founder and CEO of Wasabi, the hot cloud storage company that delivers fast, ...
In addition to 22 Keynotes and General Sessions, attend all FinTechEXPO Blockchain "education sessio...
Early Bird Registration Discount Expires on August 31, 2018 Conference Registration Link ▸ HERE. Pic...