The Toronto Blue Jays couldn't get anything going against Baltimore's relievers for six full innings. But when Kevin Gregg came on to wrap up the save, the Orioles' bullpen didn't just bend, it broke.

Pinch-hitter J.P. Arencibia hit a game-winning single with two out in the ninth as the Blue Jays rallied to beat the Orioles 5-4 on Saturday, snapping Baltimore's winning streak at three games.

"We had a chance to win the game, we were one out, one pitch away," manager Buck Showalter said. "It's a frustrated clubhouse right now."

The blown save was Gregg's seventh in 27 chances, and he's failed in three of his past six opportunities.

"Every one of them is tough, no matter how it happens," he Gregg said. "You don't like walking off the field like that."

Jose Bautista hit his major league-leading 41st home run, a two-run blast in the first, and Lawrie also went deep for the Blue Jays.

Jesse Litsch (6-3) pitched two innings for the win.

Trailing 4-3 to begin the inning, Toronto's David Cooper struck out against Clay Rapada before Showalter called Gregg (0-3) in to finish. Gregg struck out Brett Lawrie but couldn't wrap it up, walking Kelly Johnson, giving up a single to Jose Molina and hitting pinch-hitter Adam Loewen to load the bases. The closer then uncorked a wild pitch that allowed Johnson to score the tying run. Arencibia followed with a game-winning single to left.

"We got into a situation there where we didn't have much margin for error," Showalter said.

The collapse came after five Orioles relievers had combined to pitch six no-hit innings, allowing just one walk.

"The guys did an outstanding job," Gregg said. "The whole team put together a great effort and unfortunately I spoiled it at the end."

After Rick VandenHurk left in the third, rookie Zach Phillips pitched 1 2-3 innings, Chris Jakubauskas worked one inning, Troy Patton went 2 1-3, Willie Eyre got one out and Clay Rapada got two outs.

"We were definitely aware of what was going on," Jakubauskas said. "We knew that the innings were stacking up, scoreless-wise. Each guy who goes in, you feed off the last guy."

The Orioles opened the scoring with three straight two-out singles in the first, matching the number of hits they collected in eight innings against Toronto right-hander Henderson Alvarez when he beat them at Baltimore on Aug. 31 for his first major league win. Nick Markakis and Vladimir Guerrero grounded singles to center before Chris Davis lined an RBI base hit to right.

The run snapped Alvarez's scoreless innings streak at 14.

Toronto answered in the bottom half when Bautista hammered a 3-2 pitch into the second deck in left, scoring shortstop Yunel Escobar ahead of him.

An error by Alvarez helped Baltimore tie it in the second. Kyle Hudson reached on a bunt single and moved to second when Alvarez's throw sailed into foul territory down the right field line. Pedro Florimon sacrificed Hudson to third and he scored on a two-out single by Matt Angle.

Toronto promptly reclaimed the lead when Lawrie homered to left on VandenHurk's first pitch of the bottom half, his ninth.

Baltimore tied it again with another two-out rally in the third. Guerrero singled to right and scored when Davis doubled up the alley in right-center. Josh Bell followed with a single to center but Davis ran through third base coach John Russell's stop sign and was thrown out at the plate for the third out.

Robert Andino doubled to begin the eighth against Litsch, took third on a grounder and scored when Guerrero doubled to center.

Guerrero went 3 for 4 and has eight hits in has past 13 at bats.

"Very quietly, he's finishing with a little flurry," Showalter said.

Making his first start of the season after two relief appearances, VandenHurk allowed three runs and three hits in 2 1-3 innings. He walked two and struck out one.

"I just didn't throw enough strikes," he said. "That's what it comes down to."

Alvarez allowed three runs and a season-high nine hits in seven innings. He walked one and struck out four.

Escobar left in the second after being hit on the left elbow by a pitch from VandenHurk. He was replaced by Mike McCoy. X-rays were negative and Escobar is day-to-day.

NOTES: Orioles OF Adam Jones was scratched from the lineup with a sore left ankle. Jones fouled a ball of his ankle Friday night and it swelled up overnight, but Showalter said Jones might play Sunday. ... Florimon made his major league debut. ... Blue Jays OF prospect Melvin Garcia has been suspended 50 games for using a performance-enhancing substance. Garcia is the third Blue Jays minor leaguer to be suspended this season. RHP Aderly De La Cruz and LHP Luillyn Guillen received 50 games bans July 14.