Jeremy Hellickson didn't look like a pitcher making just his 11th career start.
"Hellickson, obviously, was in command," Baltimore manager Buck Showalter said. "It never really felt like you had him lined up. Didn't have many good at-bats off of him."
Hellickson (4-2) struck out three and walked one in his first career complete game.
"Once you've done it, you know you can do it in the future," Rays manager Joe Maddon said. "He was always under control emotionally."
Hellickson retired his final 13 hitters.
"I felt just as good in the ninth as I did in the first," Hellickson said. "I had a few (complete games) in high school, but this is the one that counts. This is much better."
Joyce put the Rays up with a two-run homer off Jeremy Guthrie (1-6) in the second inning. He had two hits and is 23 for 48 with five homers and 13 RBIs over his last 14 games.
Damon made it 3-0 with a solo homer in the eighth inning. He broke a tie for 73rd place on the all-time hit list with Tim Raines and Rabbit Maranville with his 2,606th hit.
Guthrie gave up three runs and seven hits in his first complete game of the season, but fell to 0-6 in his last seven starts since beating the Rays on opening day. The Orioles have scored six runs over the seven-start stretch — including no runs five times — while he was in the game.
"We know it is difficult," Showalter said. "We've talked about it. Timing is everything with something like that. He certainly doesn't want to hear it right now, I can tell you that."
The Rays improved to 10-10 at home this season and Tampa Bay is 13-5 on the road.
Baltimore was bidding for its second four-game winning streak this season.
"I thought everything was right for it tonight," Maddon said. "I'm watching him. He was not laboring to throw. He was really at a nice pace. Even though it ended up at 120 pitches, I thought he wasn't really challenged physically."
Guthrie has lost two of three starts this season against Tampa Bay, including an 8-2 defeat to Hellickson and the Rays last Saturday.
Notes: Tampa Bay CF B.J. Upton served the second game of a two-game suspension handed down by Major League Baseball following a tirade after he was ejected on May 4. ... Orioles LHP Brian Matusz, who has been on the DL all season because of a strained ribcage muscle, had a 45-pitch bullpen session. He is scheduled to start for Class-A Frederick on Monday. ... Rays LHP J.P. Howell (left shoulder surgery) could rejoin the team next Thursday. The reliever is on a rehab assignment at Triple-A Durham. ... Baltimore RHP Justin Duchscherer (left hip) threw two innings at extended spring training.