"We were living on the edge because we kept giving them chances and they came through," Berkman said.
Miguel Cairo hit a two-out, two-run, go-ahead single in the eighth inning to lift the Reds to a 5-3 rain-delayed victory against the St. Louis Cardinals.
Five Cardinal pitchers held Cincinnati to four hits but combined for five walks and a hit batter to go along with two throwing errors.
"That one inning, we made several mistakes you know one's not good and any time it is more than one it gives the other club more of an opportunity," St. Louis manager Tony La Russa said. "So, too many mistakes."
The slumping Reds had lost seven of nine before snapping St. Louis' three-game winning streak.
"This was a big win," Cincinnati manager Dusty Baker said. "We needed it badly."
Former Cardinals closer Ryan Franklin gave up a two-run single to left field by Cairo.
"Obviously, it's going to be wet but I just made an error and it happens it turned out to be a big error," Freese said. "I just got to make that play."
Aroldis Chapman (1-0) got two outs in the seventh. Francisco Cordero pitched the ninth for his third save and No. 293 for his career, tying him with Jason Isringhausen for 22nd all-time.
The game was delayed 42 minutes after seven innings.
Chris Carpenter allowed two hits over six innings, striking out six and walking two in his 300th career start for St. Louis. He had won 10 consecutive starts over Cincinnati, dating back to Aug. 15, 2006, before Saturday's no-decision.
Travis Wood started and pitched 6 1-3 innings, allowing eight hits and three runs for the Reds.
With one out in the eighth, Freese threw wildly third on a grounder hit by Drew Stubbs. Batista struck out Brandon Phillips before intentionally walking Joey Votto. Batista hit Jonny Gomes with a pitch to load the bases.
La Russa lifted Batista for left-hander Trever Miller, but Miller walked lefty Jay Bruce on a 3-2 pitch to tie the game at 3-3.
Former Cardinals closer Ryan Franklin, who lost his job after four blown saves, relieved Miller and gave up a two-run single to left field by Cairo.
"I was just trying to stay back and see what was coming," said Cairo, who turns 37 on May 5. "He left it out over the plate and I was just lucky enough to get a hit."
Franklin got out of the inning by getting pinch-hitter Jeremy Hermida to fly out. Franklin walked off the field to a cascade of boos by the fans who stuck around after the delay.
The Reds tied the game at 2-all when Votto ripped a 3-0 pitch in the left-field bleachers after Phillips walked with one out in the sixth inning.
"I got ahead of Phillips and wasn't able to put him away and fell behind Votto and made a mistake," Carpenter said. "I had a feeling he was probably thinking if he gets a good pitch to hit there he's going to swing so I'm trying to make a good pitch down and away and I cut it off a little bit and left it in the middle of the plate and he hit it out."
Albert Pujols gave St. Louis the lead at 3-2 when he hit a solo home run deep in the left-field bleachers in the sixth. The 437-foot homer tied Pujols for the NL lead with seven homers. He now has 415 in 10-year career. It was his third homer in five home games and six in his last nine.
Freese gave the Cardinals a 1-0 lead in the second with an RBI single and Colby Rasmus made it 2-0 in the third with a sacrifice fly that scored Carpenter, who led off with a double.
NOTES: Ryan Theriot had a throwing error at shortstop in the ninth, his sixth error of the season. ... Pujols and Yadier Molina received their 2010 Rawlings Gold Glove awards before Saturday's game. Sunday, Pujols and Matt Holliday will receive their Louisville Silver Slugger trophies before the game. ... La Russa said before the game Jake Westbrook will start Sunday night's series finale on three days' rest. It will be for the third time in his career he's done that and it is the first time since 2001 that Westbrook has gone on three days rest. Baker would not reveal his starter for Sunday. "It's question mark vs. Westbrook," Baker said. ... The Reds will be playing on a Sunday night for the first time for the since they faced the Los Angeles Dodgers on May 8, 2005.