NEW YORK – A low-tech telephone hot line network -- and the way it was used -- didn't do enough to send out warnings of problems on the electric power grid before the nation's worst blackout (search), power officials and politicians said.
Warnings came too late, or not at all, over the system of phones put in place to disseminate information between regional power groups designed to avert just such a crisis, officials said Monday.
Failures in Ohio transmission lines prompted at least three conversations last Thursday between FirstEnergy Corp. (search), the utility that owns them, and the industry group that manages transmission across much of the Midwest, said Mary Lynn Webster (search), a spokeswoman for the Midwest Independent Transmission System Operator.
Webster said the Midwest group initiated the first contact with FirstEnergy. The group is reviewing recordings of those conversations and computer logs and other data.
Those electricity transmission line failures, which began around 3:06 p.m. EDT, are a focus of the investigation and suspected in the blackout's possible starting point.
The failures were snowballing into system-wide disturbances by the time the Midwest group spoke with its counterpart in Pennsylvania, said Bob Hinkel, general manager of PJM Interconnection, which manages power across much of seven states and the District of Columbia.
"We know that as the specific events began to unfold we had some conversation with them about what we were observing in our system," Hinkel said. "The immediate response is you call the other operator."
FirstEnergy spokesman Todd Schneider said he could not provide any information about the conversations.
Premier Ernie Eves of Ontario complained that U.S. power managers did not notify their Canadian counterparts about the problems, as required under protocols developed after a 1965 blackout across much of the same region.
Many individual utility companies said they had little or no indication of problems in the system before their own facilities shut down.
"The first indication that we had was when we started to see our transmission lines trip out," said Ralph LaRossa, vice president of electric delivery for New Jersey's PSE&G.
A timeline of power problems compiled by the North American Electric Reliability Council, an industry group investigating the outage, showed power swings in the Eastern U.S. and Canada by 4:08 p.m. EDT and a series of line failures in Michigan starting about nine minutes later.
In Lansing, Mich., employees of the Board of Water and Light noticed irregularities on the grid, but nothing that seemed unusual. Spokesman John Strickler said any information about problems elsewhere would have helped the utility cope with the blackout, which reached its peak region-wide around 4:11 p.m. EDT.
"It looks like stuff started about an hour before this hit us. It would be nice if when the first system went down that there was an alert," he said.
Independent system operators across the country are tied together by a system of telephone hotlines. The system was put in place after the 1965 blackout to provide individual operators with vital information about events occurring outside their jurisdictions.
"It's certainly crucial that those two parties are talking to each other," Hinkel said of ties between his group and the Midwest.
Many phone lines are recorded. Those recordings, and logs of calls in and out of affected control rooms, will become key to the simultaneous investigations carried by industry groups, Congress and federal and state task forces.
"That's an integral part of this," said Stephen Allen, spokesman for the Northeast Power Coordinating Council, which is examining the failure in New York, New England and parts of Canada. "We will be looking at hardware, software and people."
The timing and content of the conversations preceding the blackout remain unclear, Hinkel and Webster said. Hinkel said the conversation between the Midwest and mid-Atlantic groups may have come seconds before the onset of the blackout.
"From the time when lines tripped in the PJM system and we lost some generation to the point in time when the collapse was all over, it all happened in roughly 11 seconds," he said. "In that period of time there's very little a human being is going to do to arrest that activity."
The Midwest power system recorded numerous voltage swings as early as midday Thursday, long before FirstEnergy's high-voltage lines failed south of Cleveland. Technicians were looking at the system reports "to determine what relationship if any (the swings) may have had with what occurred," said Webster.
The finding, if a connection can be shown, would support contentions by FirstEnergy that the system was in distress long before its lines "tripped" resulting in a reduction of power flowing through the system.
"What happened ... is much more complex than a few tripped power lines in our system," FirstEnergy spokesman Todd Schneider said. He said that the utility's data showed unusual conditions, including strange fluctuations of voltage, in the Midwest grid "as early as noon" Thursday.