B.J. Upton and Kelly Shoppach had solo homers for the Rays, losers of four of six.
Ricky Romero (4-4) allowed one run and three hits in seven innings to win consecutive starts for the first time this season. He walked two and struck out six.
Romero pitched a season-high 8 2-3 shutout innings to beat Minnesota last Friday.
Marc Rzepczynski gave up Shoppach's homer in the eighth and Frank Francisco finished in the ninth for his fifth save in six opportunities.
Rivera went 2 for 3 and scored a pair of runs as the Blue Jays won for the seventh time in eight games.
Toronto's Jose Bautista went 0 for 4 with a strikeout and failed to reach safely for just the second time in 35 games this season. Bautista had reached safely in 23 straight games, dating to April 17.
Right-hander Wade Davis gave up both of Toronto's homers and allowed seven hits in 7 2-3 innings. He walked four and struck out three. Davis (4-4) has lost consecutive starts after winning his previous four decisions.
Rivera gave the Blue Jays an early lead with a solo homer in the second that bounced off the top of the wall, his third.
The Rays tied it in the fifth when Upton homered to left-center, his seventh.
Arencibia and Eric Thames hit back-to-back singles to put runners at first and second with one out in the fifth, but Rajai Davis hit into an inning-ending double play.
After Rivera drew a one-out walk in the seventh, Arencibia whacked Wade Davis's next pitch over the wall in center for his seventh home run.
Shoppach cut the deficit to 3-2 with a solo homer to right off Rezepczynski in the eighth, his second.
NOTES: Toronto OF Corey Patterson singled in the sixth, extending his hitting streak to 10 games. ... Blue Jays 1B Adam Lind (back) is expected to miss at least another 10 days, manager John Farrell said. Lind was placed on the 15-day DL on May 16. ... Rays OF Sam Fuld got the day off, and is expected to skip Friday's game at Florida, manager Joe Maddon said. Fuld is batting .212 (7 for 33) in his past eight games.