Neuville slid the ball in with a low shot from close range after a right-wing cross from fellow substitute David Ondonkor.
A few minutes earlier, Miroslav Klose and captain Michael Ballack both hit the crossbar. Klose headed a cross off the bar in the 90th, and Ballack got the rebound and kicked off nearly the same spot.
"We had enough chances," Klose said.
Poland was playing with 10 men after midfielder Radoslaw Sobolewski was sent off in the 75th for tripping Klose.
The game between the European neighbors was scrappy and featured little flowing play, but plenty of tough challenges.
Ballack recovered from an injured muscle in his right calf that forced him to miss the win over Costa Rica and was at the center of most of his team's attacks.
Klose, who was born in Poland, had a chance in the 21st minute when he headed a curling left-wing cross past the post with only onrushing goalkeeper Artur Boruc to beat.
Lukas Podolski — another Germany striker born in Poland — had a 35th-minute shot saved by Boruc and shot wide of the post in injury time.
Maciej Zurawski and Michal Zewlakow also had shots for Poland in the first half.
Germany, which beat Costa Rica 4-2 in its opening match, now has six points but still needs Ecuador to beat or draw with Costa Rica to advance before the final group games.
Ecuador beat Poland 2-0 in its first match and meets Costa Rica on Thursday.
In other action, Spain's quest to lose its title of perennial underachiever is off to an impressive start.
The one-time semifinalists crushed Ukraine 4-0 Wednesday in Group H, getting two goals from David Villa and one spectacular strike from Fernando Torres.
Later, host Germany and Poland became the first teams to play twice at this year's World Cup when they kicked off in Dortmund later Wednesday.
Spain, which disappoints its fans more often than it pleases them, appears to be heading toward a good showing at this year's World Cup.
And the best part of its win over Ukraine was Torres' goal in the 81st.
Spain captain Carles Puyol intercepted a pass in midfield, beat a defender with a spin and passed to Torres. He sent the ball to Raul Gonzalez, who put it back to Puyol on the edge of the area. Puyol returned it into the path of Torres, who had continued running and hit the ball to the goalkeeper's right.
Wednesday's win in Leipzig helped Spain stretch its unbeaten streak under coach Luis Aragones to 23 matches.
"If we show what we can do I know we can be among the top teams at this tournament," Aragones said. "I thought it was going to be more complicated but the second goal made Ukraine more crestfallen and the rest was easier."
While the Spaniards are making their eighth consecutive World Cup appearance, they have failed to reach the semifinals since their posting their best result, a fourth-place finish in 1950.
Xabi Alonso scored Spain's first goal in the 13th minute with a header past Oleksandr Shovkovskyi.
Villa made it 2-0 four minutes later with a deflected free kick, and then added another in the 47th from the penalty spot after Torres was brought down by Vladyslav Vashchyuk, who was ejected.
"It's painful to lose 4-0," Ukraine coach Oleh Blokhin said. "I've never lost 4-0. The players did not respect the advice of their coaches. On some occasions we were playing like the worst team in Europe."
Andriy Shevchenko, who was expected to shine on world soccer's biggest stage, had little effect on the match. Blokhin had hesitated to start the striker, who was returning from a knee injury.
Jaidi scored Tunisia's equalizer in injury time, heading in Ziad Jaziri's cross from 10 meters (yards) only minutes after Sami al-Jaber had given Saudi Arabia the lead.
Al-Jaber, a 34-year-old veteran, picked up a through ball and slotted it past Tunisia goalkeeper Ali Boumnijel with his first touch in the 84th minute. The Saudi player, who came on as a substitute in the 82nd minute, has now scored in three World Cups. He also scored in 1994 and 1998, but failed to find the net in 2002.
Jaziri had give Tunisia the lead in the 23rd minute, but Yasser al-Qahtani equalized in the 57th.
Tunisia coach Roger Lemerre said he was disappointed that his team had not been able to hold on to the lead.
"We don't feel like we have opened the door," Lemerre said. "We weren't very assured. While the first goal gave us a boost, it was not enough."
Also, Wayne Rooney is healthy enough to play some part in England's next World Cup match against Trinidad and Tobago.
England coach Sven-Goran Eriksson said Rooney was not fit enough to start Thursday's Group B match, but could come on as a substitute. The Manchester United striker broke a bone in his right foot on April 29.
"For me, Rooney is match fit," Eriksson said.
Before Germany's match in Dortmund, at least 120 German hooligans were arrested after clashing with police, authorities said. At least one fan was seen receiving treatment for facial cuts.
In a separate incident, German hooligans attacked police near the jumbo screen that shows games.