With runners on first and third and the outfielders playing shallow to guard against a sacrifice fly, Gonzalez hit the winning shot off Mitchell Boggs into the left-field seats.
Albert Pujols had three RBIs for St. Louis, which is facing an uphill climb to get back to the playoffs. The Cardinals were six games back of NL Central-leading Cincinnati and 6½ back in the wild-card race before Saturday's games.
Cardinals manager Tony La Russa visited the mound but left Boggs in to face Gonzalez, who was 0 for 5 with two strikeouts before connecting for his fifth homer since he joined Atlanta in a July trade with Toronto.
Rookie Craig Kimbrel (3-0) struck out two in a perfect 12th to get the win.
The Cardinals tied it in the eighth, taking advantage when third baseman Martin Prado bobbled Pedro Feliz's grounder. Prado's one-out error loaded the bases for Pujols, who drove in Brendan Ryan with a grounder off Peter Moylan.
Pujols also had a tying two-run double in the sixth.
Georgia native Jake Westbrook pitched six innings for the Cardinals, yielding two runs and seven hits. The right-hander, who was acquired in a trade with Cleveland on July 31, walked five and struck out three.
Atlanta's Tommy Hanson gave up two runs and five hits in seven innings.
Westbrook, who walked eight batters in 44 innings over his first seven starts with the Cardinals, issued consecutive walks to Derrek Lee and Nate McLouth to force in a run in the first. McLouth added a run-scoring triple in the fifth.
Westbrook lives in Danielsville, Ga., about 90 minutes from Atlanta. He pitched only two days after his wife, Heather, gave birth to a girl, Harper, in Atlanta.
Westbrook was making his second career appearance in Atlanta, following a loss with the Indians on June 19, 2004.
NOTES: Manager Tony La Russa said LHP Trever Miller will meet the team in St. Louis after the series. La Russa said Miller, who has soreness in his left elbow, is making progress and is expecting to play catch on Sunday. ... Former Cardinals slugger Ankiel ended a streak of strikeouts in nine straight at-bats when he grounded out to end the ninth inning. ... Attendance was 51,078, the season's second-largest total at Turner Field, trailing only the opening day total of 53,081 against the Cubs.