In the nation's only other primary Tuesday, Sen. Robert C. Byrd of West Virginia handily won the Democratic race and will face businessman John Raese, who beat out five other Republicans. The 88-year-old Byrd, an outspoken Iraq war critic seeking a record ninth term, is a major target of the GOP this year.
In Nebraska, with 97 percent of the precincts reporting, Heineman received 50 percent of the vote to Osborne's 44.
Osborne was initially considered a strong favorite because of his three national championships at the University of Nebraska. In his three congressional races, Osborne never received less than 82 percent of the vote.
But recent polls showed Osborne and Heineman virtually tied and Omaha businessman Dave Nabity in single digits.
"This is a tough one to take," Osborne told his supporters.
Osborne decided not to seek a fourth term in Congress to campaign for the gubernatorial nomination.
Heineman was state treasurer when he was appointed lieutenant governor by Gov. Mike Johanns in 2001 and became governor in 2005 when Johanns was picked as U.S. agriculture secretary.
Heineman praised Osborne late Tuesday, saying: "Being involved in a race with him is very, very difficult because I have so much respect for him."
Other races of note in Nebraska included the GOP primary for U.S. Senate and the state treasurer's race between a Republican incumbent and a military hero.
In the Senate primary, Pete Ricketts, former chief operating officer of the online brokerage Ameritrade, spent $4.75 million of his own money and defeated former Attorney General Don Stenberg and Omaha lawyer David Kramer.
The winner will face Democratic Sen. Ben Nelson, who is seeking a second term.
The GOP primary for treasurer saw incumbent Ron Ross lose to Shane Osborn, who gained fame as a Navy pilot in 2001 when his EP-3 spy plane collided with a Chinese fighter jet over the South China Sea. He landed successfully with 23 crew members and then endured 11 days of captivity and interrogation.
In West Virginia, Byrd easily defeated the token opposition -- from musician and political novice Bill Hendricks Jr. -- he faced on the Democratic side.
"The trust my fellow West Virginians have placed in me is truly humbling, and you can bet I will work very hard to earn the continued support of Democrats and Republicans alike," the Democrat said in a statement.
The GOP field included Raese, a multimillionaire industrialist and media owner, and National Guard Capt. Hiram Lewis.
With 1,592 precincts out of 1,961 reporting Tuesday, Byrd had 86 percent of the vote compared to 14 percent for Hendricks. On the GOP side with 1,585 precincts reporting, Raese had 59 percent and Lewis had 23 percent.
In New Jersey, a 37-year-old former Rhodes Scholar became Newark's first new mayor in two decades. Cory Booker tromped state Sen. Ronald L. Rice in the nonpartisan race to lead the struggling city of 280,000. Seventy-year-old Mayor Sharpe James announced in March that he would not seek a sixth term.
Booker collected 72 percent of the vote to Rice's 23 percent.