Right-hander Doug Fister (3-1) allowed one run and four hits in five innings as the last-place Mariners won for just the third time in 13 games.
Reliever Shawn Kelley gave up a two-run homer to Evan Longoria, enabling the Rays to pull within 4-3 in the eighth. David Aardsma got the final three outs for his ninth save in 11 opportunities.
Tampa Bay is off to the best start in the majors at 24-11. The Rays have one of the best home records over the past three seasons, yet surprisingly are 9-7 at Tropicana Field, compared to 15-4 on the road.
Gutierrez, returning to the lineup after missing two games because of upper back stiffness, hit a two-run homer off Wade Davis (3-3) in the first inning. Adam Moore led off the fifth against the Rays starter with his first homer for a 3-1 lead.
Sweeney homered for the second straight day, hitting a solo shot off Dan Wheeler in the eighth to make it 4-1.
The Mariners entered the night with a major league-low 16 homers, but have now hit six in their past two games. They had one multi-homer game in their first 33 of the season.
Tampa Bay trimmed Seattle's early lead to 2-1 in the third. Reid Brignac doubled, continued to third on a throwing error by left fielder Michael Saunders, then scored when Fister balked. But the Rays wasted several opportunities, most notably in the fifth, when they loaded the bases with two outs and failed to score.
Fister, who walked three and struck out two, escaped the jam when Ben Zobrist flied to right on the first pitch. Mariners reliever Kanekoa Texeira worked two scoreless innings before Kelley came on and yielded Longoria's ninth homer of the season.
Davis allowed three runs and five hits in six innings. The Rays starter walked three and struck out two.
NOTES: Seattle's Ichiro Suzuki had three hits for his sixth consecutive multi-hit games, one off his career-high. ... Tampa Bay LHP J.P. Howell (strained left shoulder) threw 25 pitches in a batting practice session. He is set to pitch in a simulated game Tuesday. ... Seattle INF Jack Wilson (strained right hamstring) is scheduled to start a minor league rehab assignment with Double-A West Tennessee on Saturday. He could rejoin the Mariners in a week. ... Rays OF Matt Joyce (right elbow strain) has resumed baseball drills and could DH in a minor league game early next week. ... Rays 3B coach Tom Foley was away from the team Friday to attend to a family matter. He was replaced by bullpen coach Bobby Ramos. ... Sweeney stole third base in the first inning, the designated hitter's first steal since August 2006.