Jonny Venters offered no excuses for leaving a full-count pitch over the middle of the plate to Alex Rodriguez.
Walking Rodriguez and forcing in a run would have been a less painful option.
Rodriguez hit his 23rd career grand slam, matching Hall of Famer Lou Gehrig's major league record, and the New York Yankees scored six runs in the eighth inning to rally for a 6-4 win over Venters and the Atlanta Braves on Tuesday night.
Nick Swisher hit a tiebreaking two-run homer off Cory Gearrin two batters after Rodriguez connected against Venters.
The Braves, who wasted a strong start by Mike Minor, have lost three straight.
Rodriguez had been 1 for 10 this season with the bases loaded before he lined Venters' pitch over the left field wall. Outfielder Martin Prado barely moved as he watched the ball clear the wall.
"I made a bad pitch, and he crushed it," Venters said.
"I have no excuses. I felt great mechanically. I felt great physically. I just fell behind some hitters. ... I threw a pitch right down the middle, 3-2, to one of the best hitters in the game.
"I feel bad I let my team down."
Venters said he might have worried too much about walking Rodriguez.
"I didn't want to walk him," Venters said. "Maybe, looking back, a walk right there is not terrible, with a lefty (Robinson Cano) on deck."
The Yankees had been only 10 for 67 (.149) with the bases loaded this season before Rodriguez hit his tying shot.
Rodriguez paused to savor his shared place with Gehrig in baseball history.
"It means a lot," Rodriguez said. "It's very special. This game is very, very difficult. If you're not going to enjoy these great moments, then it's not any fun. Lou Gehrig is not only one of the all-time greats, but he's one of ours."
The Yankees matched their season high with a fifth straight win and moved into sole possession of first in the AL East when Tampa Bay lost to the Mets.
Yankees manager Joe Girardi said it was "absolutely incredible" for Rodriguez to match Gehrig's mark.
CC Sabathia (8-3) left trailing 4-0 after seven innings but was the beneficiary of the rally. The big lefty gave up four runs on a season-high 10 hits with two walks and six strikeouts.
The Yankees have won 10 of 12.
Rafael Soriano, who pitched in Atlanta from 2007-09, retired the Braves in order in the ninth for his 10th save of the season and the 100th of his career.
Minor gave up five hits and one run in 7 1-3 innings. He was pulled after giving up a one-out single to Derek Jeter in the eighth.
"You feel comfortable with a four-run lead and five outs to go," said Braves manager Fredi Gonzalez, who revealed that left-handed reliever Eric O'Flaherty wasn't available due to a sore elbow.
Gonzalez said O'Flaherty woke up Monday with the sore elbow and also wasn't available for Monday night's game. He might be held out again on Wednesday in the finale of the three-game series.
"I think we pushed (Minor) as far as we could push him," Gonzalez said. "And he did a hell of a job. It's a shame he didn't get the W."
Venters (3-3) loaded the bases, allowing a single to Curtis Granderson before walking Mark Teixeira to set up the tying grand slam by Rodriguez.
Venters, who didn't record an out, then gave up a single to Cano. Swisher followed with his 10th homer for a 6-4 Yankees lead.
Rodriguez's homer was his 10th of the season and first grand slam.
Matt Diaz hit a three-run double in the first to give the Braves the lead they kept until the eighth.
Diaz started ahead of Jason Heyward in right field for the second time in three days when the team faced a left-hander. Diaz had two hits off Toronto's Ricky Romero on Sunday.
Michael Bourn led off the first with a single and moved to third on the first of two doubles by Brian McCann. Sabathia walked Dan Uggla to load the bases.
Sabathia faced a similar jam in the seventh. With one out, Prado hit a soft single to right field and moved to third on McCann's double. Sabathia issued an intentional walk to Uggla, loading the bases for Heyward, who replaced Diaz in the seventh. Heyward drove in Prado with a groundout to second base.
NOTES: Yankees closer Mariano Rivera said that surgery on Tuesday to repair his torn right ACL "went perfectly." ... Gonzalez said RHP Brandon Beachy will be given extra rest, with his next start pushed back to Saturday, due to soreness in his right elbow. Beachy said "I feel fine" after throwing in the bullpen before the game. RHP Tim Hudson, who had his last start skipped due to bone spurs in his left ankle, is returning to start Wednesday. Following an off day on Thursday, RHP Tommy Hanson will pitch on Friday against Baltimore, followed by Beachy. ... Hiroki Kuroda, who is 1-4 with a 2.10 ERA in five career starts against the Braves, will face Hudson on Wednesday night.