Josh Johnson did his part and once again left without giving up a run.
After he left, the Cincinnati Reds got more than even.
Johnson pitched seven scoreless innings Saturday night before the Reds rallied to beat the Florida Marlins 4-3 in 10 innings.
Pinch-hitter Edgar Renteria, who holds a prominent place in Marlins' history, singled with two outs to complete the comeback.
Johnson allowed five hits and two walks while striking out six. He was pulled after 117 pitches.
"We have to win that game," said Johnson, who lowered his ERA to 0.88. "I had to battle the first three innings. I was inconsistent."
Johnson was making a bid to win his fourth game by the end of April. Dontrelle Willis was the last Marlins pitcher to reach May undefeated with at least four wins when he was 5-0 through April 2005.
Marlins reliever Randy Choate took over in the eighth with a 3-0 lead. Choate got two outs before Edward Mujica and Ryan Webb gave up three runs in the inning.
The Marlins brought in center fielder Chris Coghlan as an extra infielder, but he returned to the outfield after Ramon Hernandez grounded into a forceout at the plate. Paul Janish popped up before Renteria singled through the hole into left field.
Renteria's 11th-inning single in 1997 lifted the Marlins over Cleveland for the championship. He also was the World Series MVP last year after hitting a late three-run homer in the clinching victory for San Francisco over Texas.
"The bullpen has been so reliable," Florida manager Edwin Rodriguez said. "Mujica and Webb got ahead in the count but couldn't put them away. They had a tough night. We faced a good-hitting team. We have to give them credit."
Dunn had not allowed a run in eight innings until the 10th and nearly escaped this time. The pitcher, who came from Atlanta in the trade for Dan Uggla, took the blame.
"Almost doesn't count," Dunn said. "It was bad pitching. Free passes hurt. J.J. pitched his butt off tonight."
Francisco Cordero (1-0) allowed one hit over the last two innings to earn the win.
Control problems limited Cincinnati starter Edinson Volquez to five innings. He allowed just two hits with five walks and four strikeouts.
Volquez, who averaged 32 pitches per first inning in his first four starts of the season, showed signs of more trouble this time. He opened the game by walking Coghlan, who stole second and scored on Omar Infante's single. One out later, Volquez walked Gaby Sanchez before getting Mike Stanton to hit into an inning-ending double play.
The Marlins rode Aroldis Chapman's control problems to two runs in the eighth. Chapman walked three consecutive batters with one out before Wes Helms came off the bench and doubled into the left-center field gap.
Johnson, who hadn't allowed a hit in the first three innings of any of his first five starts, wasn't as dominant in his first start of the season against the Reds. Opposing batters were 0 for 47 with four walks and 21 strikeouts in the first three innings against Johnson before Joey Votto and Phillips came up with back-to-back two-out singles.
NOTES: Infante left the game in the middle of the fifth inning with a right hamstring cramp. ... Votto, a Toronto native, received the 2010 Tip O'Neill Award before Saturday's game. The award is given annually by the Canadian Baseball Hall of Fame to a Canada native judged to have enjoyed the best season while "adhering to baseball's highest ideals."