Shaun Suisham's 38-yard field goal with 20 victory over the Indianapolis Colts.
But Ben Roethlisberger shook off three turnovers and led the Steelers (2-1) into field goal range to finish off a 364-yard effort through the air. Mike Wallace caught Roethlisberger's only touchdown pass -- an 81-yard bomb -- and logged 144 yards on five receptions.
"I don't care how it looks, we just want to get out of stadiums with wins, particularly road stadiums, and AFC ones," Steelers head coach Mike Tomlin said. "We made the necessary plays on offense and defense and special teams."
Curtis Painter relieved Collins in the fourth and lost a fumble that Troy Polamalu returned for a touchdown. Joseph Addai ran for 86 yards and a TD for the Colts (0-3), who fell to 0-3 for the first time since 1998 -- Manning's rookie year.
Pittsburgh broke a 13-13 tie with just over five minutes left in regulation when James Harrison raced around the right end and caused Painter to fumble. The ball bounced once before Polamalu scooped it and went 16 yards the other way for the score.
Painter, who had overthrown Pierre Garcon for a potential score earlier in the fourth, orchestrated a 10-play, 80-yard response. On 3rd-and-2 from the Pittsburgh six-yard line, Polamalu came in untouched and nearly caused another fumble, but Painter got the exchange with Addai off clean, and the running back crossed the goal line with 2:09 to play.
"I thought he came back and led a nice drive and took us down the field," Colts head coach Jim Caldwell said of his third string QB.
Roethlisberger had plenty of time to march the Steelers into position for the winning kick and got a terrific 22-yard catch and run from Mewelde Moore to move into enemy territory. After two more first downs, including one on a Roethlisberger scramble, the visitors chewed all but eight seconds of the clock and ran out Suisham for the winning kick.
Wallace's 29-yard reception on the game's opening drive set up Suisham's 48- yard field goal, and the speedster beat the secondary later in the quarter to put the Steelers on top, 10-0. Roethlisberger airmailed a deep ball that traveled nearly 60 yards in the air, and Wallace did the rest.
The Steelers were on the move again early in the second, but Robert Mathis stripped an unsuspecting Roethlisberger and recovered the fumble at midfield, leading to a 21-yard Adam Vinatieri field goal.
Near the two-minute warning, Dwight Freeney blind-sided Roethlisberger just as he was about to throw, and the loose ball wound up in the hands of defensive end Jamaal Anderson, who rumbled 47 yards the other way to tie things.
Roethlisberger made it three straight turnovers when he overthrew Emmanuel Sanders on the next snap, and Joe Lefeged's interception and subsequent return allowed Vinatieri to tack on a 25-yard field goal before halftime.
Suisham hit the left upright on a 36-yard field goal try in an otherwise punt- filled third. Suisham made a 44-yard field goal nearly three minutes into the fourth for a 13-13 game.
Polamalu has four career touchdowns...Wallace has recorded six straight 100- yard receiving games...Collins was just 13-of-29 through the air for 93 yards, while Painter finished with 60 yards on 5-of-11 throws...Garcon had a team- high six receptions and 82 yards.