Minnesota is expected to pick a new lead technology contractor for its health insurance marketplace in the coming weeks. The state has been working hard to improve its website, but in its first few months serious technical problems made it difficult if not impossible to use.
Tom Baden, the state's lead information technology expert, remembers when the fear hit him. It was the end of July, only two months before the exchange, called MNsure, had to go up. Baden and his team were still waiting for a key piece of software from a vendor.
"It being so close to Oct. 1, and looking at our ability to get it in, get it installed, test it and go live ... if there was a moment where I didn't sleep a wink that night, that was my night," he recalls.
On Oct. 1, MNsure did go live but many consumers couldn't create accounts, the site would throw them off or lock up.
Becky Fink was trained to help others enroll on MNsure. But because of bureaucratic delays, she didn't receive MNsure's permission to sign people up until December. So the first day she could, she made appointments with five people.
"It never worked for me," she says. "I tried many times, many times."
After trying for about 12 hours, four of them still couldn't sign up.
"It was hard to know that I had done my best and gone through the training and always read what I was supposed to read and not be very competent," she says.
Fink's experience was common. But the full extent of MNsure's problems was hidden from the public for months.
Three weeks into the website's bumpy roll-out, April Todd-Malmlov, MNsure's executive director, declared all was well.
"At this stage, I think the website is doing a very good job," she said to Minnesota Public Radio. "Does that mean it has everything in it that we ultimately wanted to have? No. Our goal is to continuously improve it over time."
But an independent review in January found that Todd-Malmlov and other MNsure leaders were in crisis mode after Oct. 1. The site was buckling under the onslaught of traffic, consumers flooded the call center and hourlong waits were the norm.
April Todd-Malmlov resigned in December.
What Minnesotans did not know is they were testing the site. There wasn't time for consumer testing before the site went live.
Michael Krigsman, a consultant who specializes in diagnosing and preventing IT project failures, says testing is key.
"That is so screwed up. You can quote me on that," he says. "This is one of these things that's so foundational. It's like why do we need to breathe the air?"
But the lack of consumer testing is just one reason why MNsure sputtered. The state had trouble deciding on a lead contractor to build MNsure, which burned up valuable testing time. And in early 2013, the state had to make a major adjustment after the federal government said exchanges had to meet 70 performance benchmarks. On top of it all, Minnesota had adopted one of the nation's most ambitious plans for its marketplace in the first place.
Dannette Coleman, who runs the individual and family business lines for Medica health plans, sees another problem: People with expertise in health policy, not IT, largely directed the project.
"Incredibly dedicated and committed people who really believed strongly in the work," Coleman says. "But they didn't really have the background to understand what it was going to take to get this project done on time, on budget."
In hindsight, MNsure's board of directors say they should have demanded more information from Todd-Malmlov. They were comfortable with her only delivering reports orally before the board.
MNsure's board chair chairman Brian Beutner now regrets the agency wasn't more forthcoming about the system. "I think if I could point to one of the largest failures of MNsure, it's been a communication failure," he says.
"It's been managing the expectations of what was actually being built, when it was going to be delivered and what was that functionality."
Meanwhile, the MNsure site limped along and sucked up thousands of staff hours fixing problems and developing workarounds.
On New Year's Eve, the last day to sign up for coverage that would take effect the next day, Becky Fink called in the four people she had tried and failed to enroll through the MNsure website.
"They all came in and I gave them all coffee, and we made copies of the paper applications and we got them all done and we faxed them in," she says.
The state has spent about $100 million so far on creating its exchange. State officials had expected MNsure to enroll up to 1.3 million people for insurance by 2016, but a recent review by consulting firm Optum found MNsure is unlikely to meet those goals.
MNSure leaders will decide soon whether to build a new website or try to improve on the current site. But officials say it is stable enough now to handle an expected increase in traffic in the next few weeks. The deadline to sign up for health insurance this year is March 31.