Travis Snider went from total frustration to utter elation.
The decisive hit came two at-bats after Snider struck out swinging to leave the bases loaded, and responded by breaking his bat over his thigh and angrily slamming the pieces into the dirt around home plate.
"That wasn't a great demonstration of professionalism," Snider said of his outburst. "Sometimes as competitors our emotions get the best of us. I'd like to call it a relief but it's not one of those things that you want to make a regular occurrence."
The Blue Jays scored twice off Yankees closer Mariano Rivera in the ninth to tie it before winning it against Ivan Nova (1-2).
"That's all the inspiration we need as players is to see what we did in the ninth inning against Mariano," Snider said. "To be able to give ourselves a chance, to go out there and just have that shot, I'm very thankful for it."
Edwin Encarnacion opened the 10th with a single. After Jayson Nix and Yunel Escobar flied out, Snider drove a double to right-center. Snider, who went 1 for 6, tossed his helmet away in celebration as the winning run scored.
"We all saw the frustration," Toronto manager John Farrell said. "But to his credit, he didn't cash in the night."
Jon Rauch (1-1) pitched one inning for the win as Toronto snapped a three-game losing streak and won for the first time this season when trailing after eight innings.
Jose Bautista hit his fourth homer for the Blue Jays, a solo drive.
Rivera had converted all seven of his save opportunities this season.
"We got it to where we wanted to and we just didn't hold onto it," Yankees manager Joe Girardi said.
Escobar led off the ninth with a double to center, took third on a grounder and scored on a wild pitch that was also ball four to Bautista.
"I held the ball too long," Rivera said. "I wanted to go away and it was way too away and ended up being a wild pitch. I was upset with that one."
Adam Lind followed with a single, putting runners on the corners for John McDonald, who drove in the tying run with a squeeze bunt up the first-base line. Teixeira charged the ball and flipped to catcher Russell Martin, but it was too late to get the sliding Bautista.
"It's just a great play," Teixeira said. "That's why you run it because when it's run well, it's almost impossible to defend. I thought there might have been a chance to get him so I gave it a quick flip but it was a good play."
Jose Molina followed with a single, loading the bases for Corey Patterson, who grounded into a double play.
The rally snapped Rivera's streak of 17 1-3 scoreless innings in April, a run that stretched back to April 30, 2009.
"That's the thing about Mo, he's not perfect but he's close to it," Teixeira said.
Bautista got the Blue Jays started with a drive off A.J. Burnett in the first, but the Yankees answered with a two-run second. Nick Swisher and Eric Chavez opened the inning with walks, Jorge Posada hit an RBI single and a second run scored on Martin's double-play grounder.
Toronto tied it in the third, taking advantage of a throwing error by Burnett. Bautista drew a one-out walk, took second on a wild pitch, then escaped from a rundown and advanced to third after Jeter couldn't handle Burnett's throw on Lind's comebacker. Aaron Hill followed with a sacrifice fly.
After Granderson singled to open the sixth, Teixeira followed with a drive to center.
Toronto got one back in the bottom half. Hill singled and stole second but had to leave with tightness in his right hamstring, with John McDonald coming on to run. Jose Molina flied out and Corey Patterson walked before Encarnacion hit an RBI double.
Burnett left after walking Nix to load the bases, but David Robertson fanned Escobar and Snider to escape the jam.
Yankees third baseman Alex Rodriguez was held out of the starting lineup for the second straight game because of a sore left side. Rodriguez said he expects to return Wednesday.
NOTES: New York also leads the majors in multihomer games (10). ... Toronto activated RHP Frank Francisco (right shoulder) off the 15-day disabled list and optioned RHP Jesse Litsch to Triple-A Las Vegas. ... Yankees RHP Carlos Silva threw 40 pitches during his second bullpen session at New York's spring training complex in Florida and could face hitters early next week. Silva, released by the Chicago Cubs in spring training, agreed to a minor league deal with New York on April 9. ... Yankees C Francisco Cervelli, coming back from a broken left foot, caught Silva. Cervelli is playing in extended spring training games.