DETROIT – Jerry Hoisington thought he was seeing things: In the blink of an eye, he watched from his computer screen as southeast Michigan became powerless.
Hoisington was at the controls of DTE Energy's (search) system operations center Thursday when the blackout of 2003 swept through the Midwest and Northeast, leaving millions in eight states and Canada suddenly without juice.
About 4:10 p.m., the lights blinked in the operations center — which is reminiscent of a NASA mission control center — and DTE's own generator kicked on. For a moment, Hoisington's computer locked up. Then it displayed the unbelievable.
"All of our generators were off except for two in St. Clair [County]," Hoisington said Monday. "We have 25 generators. All I could think was, `This is really bad."'
In seconds, DTE Energy's 2.1 million customers, who live and work in eastern Michigan, were without electricity.
Hoisington's phone started ringing. One of the first calls was from Ron May, DTE Energy's senior vice president for energy distribution, who wanted to know what was happening.
"I told him I thought we'd lost the system," said Hoisington, who's worked at the utility for 19 years. "I couldn't believe it. He couldn't believe it. He said, 'I'll be right down."'
Another call. This one from Van Greening, who manages the operations center. Greening was in his car when his mother called and said she'd lost power and water at her home.
"Jerry indicated that we'd lost the entire system," Greening said. "I said, 'Repeat that. What do you mean we've lost the entire system?' It's something we train for, but never in our wildest dreams do we think something like this could happen."
When the nation's electrical grid works properly, individual power failures seal themselves off so electricity keeps flowing. When the grid fails, the often tenuous supply-demand balance between power plants and consumers can spur massive surges one way or the other, shutting down substations and generators as they try to protect themselves from damage.
That's just what happened Thursday at DTE Energy and at Consumers Energy Co. (search), the Jackson-based utility with 1.6 million customers. About 100,000 Consumers Energy customers lost power; nearly as many customers of the Lansing Board of Water & Light (search), a municipal utility, also were left without power.
Within 90 seconds of seeing the voltage drop, Board of Water & Light's system was dead. Alarms were sounding, and the computer screens telling the controllers what was going on were scrolling red type — lots of red type.
"There was a lot of chaos because all the generators tripped off and all the alarms tripped off," said Vearl Church, a system control supervisor and a 41-year veteran of electricity management.
Church called Consumers Energy officials, but all they could tell him was they were in trouble, too.
"We had no warning. Even afterward, we had no idea what caused it," Church said. "I didn't know the extent of the outage until quite a bit later, because I just didn't have time to find out about it."
Steven L. Ray, Consumers Energy's executive manager for system operations, was working in the company's Grand Rapids control center Thursday afternoon when he was called into the control room.
Something was amiss.
Voltage was dipping. Within seconds, four of the company's generating units tripped off line. The utility had lost 1,700 megawatts — or about 20 percent of its capacity — and 100,000 customers on the outskirts of Detroit were powerless.
"The automatic relaying and protection devices saw the tremendous power flows and voltage swings and automatically worked to stabilize the system," Ray said. "And the operator watched."
Ray said he and others had seen voltage sags before, so at first they weren't aware of the enormity of the situation.
"Then it became apparent that it wasn't just an isolated area," he said. "Once that happened, the realization hit that this was a very large event and was going to be something significant."
Board of Water & Light customers woke Friday to find their lights and air conditioners working again. By midday Friday, Consumers Energy had restored power to nearly all affected customers. Early the next morning, DTE Energy did the same.
Greening said DTE officials are reviewing records of the system's activities leading up to the blackout and its reaction to the crisis. Those eventually will be turned over to state and federal groups investigating the blackout.
"There's a long record, and we're poring over that data right now," Greening said.