The meeting Friday afternoon at a golf club in the capital began hours after the two sides decided to begin negotiations.
Before the meeting got underway, Home Minister Krishna Sitaula told reporters that there were "no differences" between the parties.
"The talks should be successful," said Sitaula, one of three ministers in the government's delegation to the talks.
Nepal's new government took office after King Gyanendra agreed to relinquish control over the country last month following weeks of protests and a general strike.
The unrest was organized largely by politicians now in power and backed by the rebels — helping to forge a link between the new government and the insurgents that has clearly made early moves toward peace relatively painless for both sides.
The government has released hundreds of rebels from jail, dropped terrorism charges against them, and agreed to a cease-fire. It also has agreed to rewrite the constitution, a key rebel demand that crippled peace talks in 2001 and 2003.
The rebel delegation to Friday's talks was being led by Krishna Mahara, who earlier in the day met with Prime Minister Girija Prasad Koirala to decide on an agenda and code of conduct for the talks, officials said.
There was no immediate word on how long the talks would last, or even if they would continue past Friday.
The rebels, who claim to be inspired by Chinese revolutionary Mao Zedong, began fighting to replace the constitutional monarchy with a communist state in 1996.