Updated

The road back to the playoffs is getting bumpier for the Tampa Bay Rays.

Russell Martin's three-run homer highlighted an eventful early burst and the New York Yankees kept their AL East lead with a 6-4 win Sunday that ended a most damaging trip for the Rays.

Minus ejected manager Joe Maddon, the Rays fell five games behind the Yankees and still trail the Orioles and Los Angeles Angels in the wild-card scramble. Cleanup man Matt Joyce also was tossed as Tampa Bay finished 1-5 on its visit to Yankee Stadium and Camden Yards.

"The math isn't in our favor," Maddon said.

Then again, Tampa Bay pulled off an incredible rally last season to earn a playoff spot on the final day. This year, an extra slot is available.

"The baseball gods have created two wild cards, so let's see what happens," Maddon said.

Ben Zobrist homered and doubled for Tampa Bay. Down 6-1 in the sixth inning, the Rays later brought the potential go-ahead run to the plate but couldn't break through against the back end of the Yankees' bullpen.

"It seems like we're pressing now," Rays star Evan Longoria said.

Hiroki Kuroda (14-10) struck out 10 in six innings and beat Tampa Bay for the second time this month. Rafael Soriano, the Yankees' fifth pitcher, got his 40th save in 43 chances.

Matt Moore (10-11) lost his fourth straight start. The rookie and Kuroda started out well in the last scheduled game between the teams, with the first nine outs overall on strikeouts. Each team finished with only five hits.

Martin's opposite-field shot to right field came during a crazy five-run third. The inning included an errant pickoff throw, a wild pitch, three walks, a pair of steals, Nick Swisher's first sacrifice bunt of the season, warnings to both benches and Maddon's ejection.

"That third inning was just atrocious," Moore said.

Derek Jeter put the Yankees ahead with an RBI single, moments after playfully pointing his bat toward a young fan who was taking his picture while he was on deck. Alex Rodriguez clapped his hands after singling through the drawn-in infield for a 2-0 lead.

Martin made it 5-0 with his 17th homer. Two batters later, Moore threw a 95 mph fastball behind the helmet of Curtis Granderson and plate umpire Paul Emmel immediately warned each side against any trouble.

Maddon had a few words for Emmel.

"I kept saying that's not necessary," Maddon said. "I said, 'You're wrong.'"

Once he was ejected, Maddon got tossed and came out for a protracted discussion with Emmel. Granderson crouched outside the batter's box during the dispute, watching the antics.

The wild inning seemed to wreak havoc with the scoreboard, too. As the Rays hit in the fourth, the score on the main board changed from 5-0 to 4-0 to 2-0 to 0-0, Kuroda's pitch count went in reverse and innings were wiped off.

Eduardo Nunez stole three bases for the Yankees and set up Rodriguez's sacrifice fly in the fourth for a 6-1 lead. Robinson Cano followed with a routine fly that left fielder Desmond Jennings lost in the sun for an error.

The Rays rallied for three runs in the sixth, helped by a couple of lucky bounces.

Zobrist walked after his foul pop fell out of fill-in first baseman Steve Pearce's glove at the railing, then Longoria's possible double-play grounder took a bad hop over Rodriguez's shoulder at third base for a two-run single.

NOTES: Rays RF Sam Fuld exited with a mild right hamstring strain after grounding out to close the fifth. 2B Cano bobbled the ball, recovered and threw out Fuld, who grabbed the back of his leg after crossing the bag. Maddon said he expected Fuld to miss at least a day or two. ... Maddon has been ejected four times this year and 27 times as Tampa Bay's manager. Bench coach Dave Martinez ran the team with Maddon gone. Martinez recently interviewed for the job as Houston's manager. ... B.J. Upton of the Rays struck out all four times up. ... Rodriguez stole his 12th base in 13 tries this year. ... The Rays won the season series 10-8. ... RHP Alex Cobb (9-8, 4.26 ERA) and the Rays host RHP Aaron Cook (3-10, 5.18) and Boston on Monday night.