Robert Andino's game-winning single in the 4 victory over the Yankees in a makeup game at Camden Yards.
The contest had been previously rained out twice, once on August 22 and again on August 27 on account of Hurricane Irene.
Andino finished with two hits and two RBI, including the game-tying base hit in the eighth inning.
Clay Rapada (1-0) recorded the final out in the top of the 10th inning to hand Baltimore its second straight victory in its last at-bat.
Scott Proctor (0-1) tossed a scoreless ninth before allowing the game-deciding run in the 10th.
Eric Chavez and Derek Jeter each knocked in two runs for New York, losers of two straight after a six-game win streak. The Yankees remained 2 1/2 games ahead of second-place Boston in the AL East after the Red Sox lost in Toronto Thursday night.
Alfredo Simon allowed four runs on six hits and a season-high six walks through a four-inning start for Baltimore, and Ivan Nova allowed three runs on five hits and three walks through 5 1/3 innings for New York.
Nolan Reimold singled with one out in the home 10th and Chris Davis followed by working a walk on an eight-pitch at-bat. Andino stepped up next and pulled the second pitch he saw down the third-base line, easily scoring Reimold from second for the win.
The Yankees loaded the bases with no outs in the second on walks to Alex Rodriguez and Nick Swisher around a base hit by Robinson Cano. After an infield single by Chavez plated Rodriguez, Francisco Cervelli hit a broken-bat liner to third that Davis grabbed and tossed to second, catching Swisher off the bag for a double play.
Brett Gardner worked a walk to load the sacks a second time before Jeter singled up the middle to bring Cano and Chavez across the plate, giving New York a 3-0 lead.
Baltimore got one of those runs back quickly as Vladimir Guerrero led off the home half with a home run over the left-field wall, but Chavez lined a run- scoring base hit in the third to give the Yankees back their three-run lead.
"We didn't score after the third inning," Yankees manager Joe Girardi said. "That's what kind of hurt us today. They kept chipping away. I didn't think Nova was as sharp today."
Guerrero led off the fourth with a double high off the right-center field wall, advanced to third on a Matt Wieters groundout and crossed the plate on Mark Reynolds' grounder to second to make it a 4-2 game.
After a one-out double by Reimold in the sixth inning, Nova was pulled in favor of Boone Logan. Davis greeted Logan by hammering a double into right-center, bringing home Reimold for a one-run game.
With a man on first in the bottom of the seventh, Guerrero laced a double into right-center field. Markakis rounded third on the play, trying to score the tying run. Curtis Granderson threw to Cano, and his relay beat Markakis to the plate and Cervelli held on to the ball as Markakis plowed into him.
"I don't think I've seen a better relayer of the baseball than Cano," Orioles manager Buck Showalter said about Cano's throw to home.
Despite having another run thrown out at the plate in the eighth, the O's managed to tie the game on Andino's RBI single through the left side of the infield.
Despite the setback, the Yankees won the season series between the two clubs, 13-5, winning six of nine at Camden Yards...New York fell to 38-11 in day games this year...Nova has not lost since June 3 but saw his streak of eight wins in eight starts halted.