When Healthcare.gov launched, it drew an understandably high number of initial users. Millions poured onto the site, but they weren’t able to sign up for insurance due to technical glitches. As an impartial observer, it was interesting to watch media outlets struggle to find even one person that was able to sign up successfully. The Washington Post even went as far as to illustrate this single newly minted healthcare insurance holder as a mythical unicorn.
Leave it to clashing political tensions to throw the topics of non-functional requirements, project management, and user experience into the limelight. Oh, wait. That’s not what everyone has been talking about since the wake of the Health Insurance Marketplace ribbon cutting…but they should.
There’s lots of finger pointing in the great game of Healthcare.gov Whodunit. However, underneath all of the tensions that bely healthcare reform, there are some key takeaways from the Healthcare.gov case study for anyone looking to build a website as a platform for information dissemination and conversion. Here they are:
Project Management
It was originally thought that there were only two players involved in the creation of Healthcare.gov. In reality, there were more than I have fingers to count with. A project this colossal requires some serious project management, and project management was clearly lacking here. It has been reported that those in charge were aware of the flaws and were told the site was not ready for launch. The Washington Post reported that “people were pulling out their hair” and complaining “loudly” about the problems the site was experiencing before being moved over to the live server. Those in charge still insisted on rolling out the new site on the original timeline.
This brings us to Lesson 1 in PMP 101: The Project Management Triangle. All projects need to be performed and delivered under certain constraints. These are illustrated as a triangle of “scope” or “quality” vs. “time” vs. “cost”. These three are inextricably tied, and one side of the triangle cannot be changed without impacting the other two sides. In this case the triangle failed, and quality suffered as a result.
Lesson for the Rest of Us: I’ve worked on projects that have had multiple organizations working together towards a common goal. This always requires the requisite oversight and planning according to scale. Sometimes a client may want to forgo a project manager role in order to decrease overall project cost. If you want to save costs, then you do so by having good project management. The reverse does not work.
Testing & QA
The biggest impact the imbalanced Project Management Triangle had on the Healthcare.gov site was that “time” had to be given up since the work stream was not adequately scoped from a quality perspective. This ultimately left no time for the most important task of all: testing and bug fixes.
Let’s bring in an example. There were reports that individuals were having issues during the process of signing up for a username and password. User attempts at this fundamental process have resulted in users that are now committing unintentional polygamy with multiple spouses attributed to them by the federal government. More times than that, the site is just freezing and users are not sure if they should wait, hit the back button, or just hit the submit button again and again and again.
This is fundamental. Nothing else works without being able to sign up. As a result, a team of QA specialists should have gone in and tested the sign up process on multiple versions of Internet Explorer, Firefox, Chrome, etc. They should have made sure it worked for PC and Mac users. They should have tested under different user scenarios (single user, versus married user, versus family). Instead, they didn’t do it at all. I was reading this bit on CBS news on the topic of Healthcare.gov and testing. As stated in an interview with Luke Chung, an online database programmer:
It’s not even close. It’s not even ready for beta testing in my book. I would be ashamed and embarrassed if my organization delivered something like this.
Lesson for the Rest of Us: Testing a site after launch is like ” trying to repair a car while someone is driving it.” I know that the launch deadline always has everyone on the project team on edge. No one wants to be the one that asks for more time. However, if something is worth being done, then it is worth being done right. There’s a reason that carpenters measure twice before cutting.
User Experience
Central to the Healthcare.gov debacle is the importance of user experience. The healthcare insurance buying process is complicated. Much more difficult than, say, buying a book on Amazon. So, why is it that more effort is being put into journey mapping the experience of book buyers than were given to the experience of anxious healthcare insurance consumers going through a federal process for the first time, ever?
While “non-functional requirements” (more on that below) were initially blamed for the Healthcare.gov shortfall, the most significant issues were related to user experience. It was this front end that was reportedly: 1) tested least, 2) not done correctly, or 3) both. My bet is on 3. The fact that millions of insurance consumers that had set up accounts are now being asked to sign up again points to significant changes being made to the fundamental components of the site. From a coding perspective, this is like a complete front end rebuild.
Lesson for the Rest of Us: Don’t let a complete front end rebuild after launch happen to you. As I mentioned in the post “Would you co-design you website with patients?“, users are the reason you are building the site in the first place. As a result, they should be at the table during the site design. Many of the healthcare organizations that I work with already have a patient advisory council of some sort that plays a role in governance. Councils like these are perfect contributors for user experience surveys and usability testing.
Non-Functional Requirements
I end where this whole debacle began. White House officials originally blamed high volume, with more than 8 million hits to the site in the first week, as the reason for the problems with Healthcare.gov. Even if this were the actual reason for the problems, which it wasn’t, building out non-functional requirements would have easily uncovered the hardware needs for a site this massive to ensure it did not collapse.
For the uninitiated, non-functional requirements are the requirements related to the nuts and bolts needed to make a site function behind the scenes. Among other things, it studies site usage and then the hardware needs required to support that usage. Healthcare reform involves multiple government agencies, numerous private insurance companies, all 50 states (the majority without a state health insurance exchange of their own). This site had to hold countless legacy computer networks together with more than duct tape and twisty ties. My original reaction to the number of users was “only 8 million hits?” In the grand scheme of things, that’s not a ton. Even back in 2000, there were 50 million Americans and their dependents who were self-insured. This doesn’t even count the millions more that are uninsured. Non-functional requirements take things like that into account.
Lesson for the Rest of Us: Organizations are, usually, good about building out the functional requirements before developing. Non-functional requirements are often left up to educated guess. “Gut instinct” is the lazy man’s non-functional requirements. Don’t do it.
Anything you think I missed? Please comment below.
