Pittsburgh (13-4) will play the winner of Sunday's game between the New England Patriots and the New York Jets for a spot in the Super Bowl, at Dallas on February 6.
Rashard Mendenhall scored the game winner on a two-yard run with 1:33 left, the gritty running back powering up the middle while taking several Ravens with him into the end zone.
"Anytime you can beat them it feels pretty good," said Steelers quarterback Ben Roethlisberger, who completed 19 of 32 passes for 226 yards and two touchdowns. "But to do it with the last drive we had is always special."
The Steelers trailed 21-7 but took advantage of three third-quarter turnovers by the Ravens to take a 24-21 lead in the final period on a 35-yard field goal by Shaun Suisham.
"We had concerns going in to this thing on how the game would start," said Steelers coach Mike Tomlin. "They did start fast. We got behind. But the guys didn't blink.
Baltimore scored its only points of the second half when Billy Cundiff booted a 24-yard field goal with just under four minutes left to tie the score at 24-24 before Mendenhall's second touchdown of the game sealed the result.
The big play in the Steelers' game-winning drive was a 58-yard pass from Roethlisberger to Antonio Brown on a third and 19 from the Pittsburgh 38.
Brown's catch down the right sideline gave the Steelers a first and goal from the four and on a third and two Mendenhall scored to send the fans at frigid Heinz Field into a frenzy.
Pittsburgh outgained the Ravens 263-126 in total yards but it was the third quarter turnaround that has the Steelers in the conference championship game.
Ravens running back Ray Rice lost a fumble and quarterback Joe Flacco had an interception and a fumble that let the Steelers back into the game. The Ravens' miscues opened the door for the Steelers, who scored 17 points on the turnovers.
"They know how to win playoffs games," said Ravens linebacker Terrell Suggs, who had three sacks of Roethlisberger. "That's a veteran team over there.
Baltimore and Pittsburgh each had 12-4 records during the regular season but the Steelers won the AFC North in a tiebreaker. The teams split the season series, each winning by three points.
Tomlin likened Saturday's game to a bout between two great fighters.
"Styles make fights," he said. "Those two teams, that's Hagler-Hearns right there. It's great for the game of football."
(Writing by Steve Ginsburg in Washington, editing by Julian Linden)