Updated

CC Sabathia and the New York Yankees were in complete command for seven innings, until Rafael Soriano and the rest of the bullpen threw all that hard work away.

Joe Mauer hit a go-ahead single in the 10th and the Minnesota Twins broke through in the Bronx, rallying past New York 5-4 on Tuesday night to overcome a dominant outing by Sabathia.

"Just one of those nights. It's going to happen. But I have full confidence that these guys are going to shut the door nine times out of 10, and that's really the strength of our team," Sabathia said.

For the second night in a row, the Yankees hit a first-inning homer and opened a 4-0 cushion after two. They held on 4-3 in the series opener, but couldn't do it again.

"We've got to pile on. We can't be satisfied with four runs in the first two innings," said Mark Teixeira, who hit a three-run homer.

Delmon Young blooped a three-run double in the eighth to tie it and the Twins took advantage of five walks by Yankees relievers in the last three innings. Matt Capps (1-0) got six straight outs and Joe Nathan earned his second save since missing last season following Tommy John surgery on his elbow.

"It was a good win for us to battle back against this team in this stadium," Nathan said. "Tough conditions throwing and hitting. It seemed almost like we were throwing ice cubes, it was so cold and dry."

The comeback had to be especially satisfying for the Twins. Including playoff games, they are 7-31 in the Bronx since Ron Gardenhire took over as manager in 2002. New York has won 18 of the last 21 meetings overall, counting three-game sweeps in the AL division series each of the past two years.

"It's never easy to play them, no matter where the game is. And even when you get a lead, those guys know how to win," Gardenhire said. "Our boys showed a lot of courage. It was a great win."

Teixeira hit another three-run homer to extend his exceptional start and Andruw Jones went deep his first time up with the Yankees, who appeared poised to torment the Twins yet again before the bullpen collapsed.

Sabathia tossed seven shutout innings, retiring his final 17 batters. The big left-hander yielded just a pair of second-inning singles before a light crowd on the latest chilly night for baseball in the Bronx.

He struck out six, walked one and was pulled after 104 pitches. But the Yankees couldn't preserve a four-run lead.

Soriano, spotless in his first two outings after signing a $35 million, three-year contract, walked three in the eighth. The last one, to Mauer, forced in a run and when Young plopped a three-run double into right off David Robertson, the Twins had fought back to tie it.

"If Mariano Rivera can blow a save every now and then, Soriano can have a rough night," Teixeira said.

Denard Span drew a leadoff walk from Boone Logan (0-1) in the 10th and Tsuyoshi Nishioka fouled off two bunt attempts before hitting a single to right that sent Span to third. Mauer lined a single to right through a drawn-in infield.

Minnesota starter Brian Duensing quickly fell behind 4-0 in his season debut, continuing a frustrating trend for himself and the Twins at Yankee Stadium.

Duensing lost in New York in both division series losses, unable to get through the fifth inning in either outing. This time, he settled in after a rocky start and went seven, keeping Minnesota within striking distance.

Danny Valencia walked to start the eighth, pinch-hitter Jim Thome worked a free pass with one out and Span singled to load the bases. After Nishioka struck out looking, Mauer walked for his first RBI of the season.

Robertson came on and Young looped a 3-2 delivery toward the right-field line. The ball dropped just in front of a sliding Nick Swisher and squirted past him, allowing all three runners to score.

"We were in a no-doubles situation. It was an aggressive mistake and we paid for it," Swisher said. "I thought I was going to make the catch, I really did. I was flying in there and the ball just kept fading away."

Derek Jeter drew a leadoff walk in the first and Swisher singled before Teixeira pulled a 2-0 pitch into the left-field stands for his fourth home run in five games this year. It was his third three-run shot, and the first time he's connected from the right side.

In the second, Jones hit a liner into the front row of left-field seats for his 408th career home run. A former star with Atlanta, he signed a $2 million, one-year deal in the offseason to be New York's fourth outfielder.

NOTES: Twins RHP Carl Pavano, hit hard in an opening-day loss at Toronto, is set to start against his former team Wednesday night. RHP Freddy Garcia makes his Yankees debut. ... Announced attendance was 40,267 on a 45-degree night, but there were plenty of empty seats. It was the smallest attendance in the three-year history of new Yankee Stadium, surpassing Monday night's crowd. ... With Jones starting in LF, slumping Brett Gardner was left out of the New York lineup.