Most Read This Week
Why Obama Administration Should Have Paid More Attention to Load Testing
What needs to be understood here is that it’s important to test early and often
By: Tim Hinds
Feb. 17, 2014 09:45 AM
October 1, 2013, was the most anticipated date for the Obama administration since his re-election. It was to be the day every American would have access to health care on one centralized website. However, according to at least one report only six people enrolled in Obamacare on the first day. Then shortly after, the entire website crashed along with its infrastructure.
The massive crash happened because within the first 10 days of launch HealthCare.gov had over 14.6 million unique views. Something the Obama administration was not prepared for, nor the testers.
The website should have been able to handle tens of thousands of people at once, but in a trial test before the launch a mere 500 users caused the website to crash. In testimony before U.S. Congress, the contractors responsible for HealthCare.gov said they didn't have enough time to fully test the website. The inability to properly load test the website well before the launch date of October 1st led to one of the worst federal website debacles of all time.
What Went Wrong
The site was built by 55 contractors and is considered one of the most complex software projects ever undertaken for the federal government, which might be where their problems all started.
According to Louis Woodhill, a contributor to Forbes magazine, the Obamacare website is comparable to the Soviet Union. "In their effort to build an IT system to implement Obamacare, the U.S Department of Health and Human Services was trying to do the same thing as the USSR's Gosplan agency: elicit coordinated, purposeful action from a collection of entities that don't know each other, don't trust each other, have conflicting objectives, and face diverging incentives."
Mixing contractors wasn't their only issue, the Obama administration continued to make a series of rookie mistakes that led to the demise of the website.
Incorrectly Assessing User Behavior. First, the administrators in charge of the website decided in late September to exclude the feature that would let people shop for health plans before registering for an online account. This lead to a bottleneck in the process because more people than expected had to go through the registration process before they could even browse through plans.
Broken Systems Integration. Second, the registration process was flawed. The consumer was supposed to enter basic account information, a security question and so on, but the communication between the systems responsible for storing this information wasn't working properly. This resulted in thousands of users who were unable to successfully create an account.
Rebuilding Components from Scratch When Proven Systems Were Available. Last, the Data Services Hub, which is a proven identity service available to the government for consumer applications, was surprisingly not used to its full extent. Instead, the website builders created new software systems meant to do exactly the same thing. In an article by Mashable the author emphasizes the fact that if the HealthCare.gov site had in fact fully leveraged the Data Hub, then it wouldn't have been such a mess.
With all of these missteps and rookie mistakes under consideration, what is known is the fact that HealthCare.gov was overwhelmed with the amount of visitors to one site.
Why the Government Should Have Made Load Testing a Priority
Prioritization of Problems and Fixes
The major pain point in the entire HealthCare.gov website was the registration process that millions of Americans attempted to fill out. Had they load tested the website months out from the launch, the team would have been able to identify the root causes of performance issues and determine whether they were in application code or the app servers and infrastructure components.
Earlier Identification of Issues
This chart illustrates how much it costs the paying client to fix a bug according to the stage of development. At the operation stage, a bug can cost clients more than 150 times as much as a bug caught in the requirement stage.
Had the testers broken down their tests into smaller test cases, over time the administration might have taken the time to listen and understand that these little bugs needed to be fixed prior to the public launch.
Decisions Made from Intelligence on the Ground
The administration decided to completely ignore the classic project management triangle.
The only way to increase the scope of a project without changing the due date would be to add more resources. Since the administration was rigid on all three sides of the triangle, the quality of the website suffered.
It's no wonder this website failed. The dynamics between the testers and heads of HealthCare.gov were strained, and it appeared the Obama administration chose to ignore testers who knew the website was not ready.
Apparently, federal workers aren't able to access consumer data manually. "An unknown number of customers who are trying to get help through less formal means - by calling the health care marketplace directly - are told that HealthCare.gov's computer system isn't yet allowing federal workers to go into enrollment records and change them."
What needs to be understood here is that it's important to test early and often. If tests would have been conducted throughout the entire website development, the Obama administration would have avoided such an embarrassing and reputation-tarnishing event.
Reader Feedback: Page 1 of 1
Subscribe to the World's Most Powerful Newsletters
Today's Top Reads