The report Tuesday to the U.N. Security Council said there are thousands of documented cases of women and children abducted for forced labor or forced sex. There are also dozens of cases of children killed in fighting in southern Sudan and Darfur, it said.
The report also examined what Annan called the "deeply rooted" practice of recruiting child soldiers in southern Sudanese military culture, a practice that has taken place as recently as last month.
"It is estimated that thousands of children are still associated with armed forces and groups in Darfur and were actively involved in conflict between May and July 2006," it said.
Official military groups such as the Sudan Armed Forces and the Sudan People's Liberation Army, as well as militias like the Janjaweed are only some of the groups responsible for the thousands of children who are "used in conflict," according to the report.
These groups are also responsible for child abductions and sexual violence that plague the country and often go unreported, it said.
Annan said he is concerned at the human rights violations against children, which largely go unpunished, and he urged both the Sudanese government and rebel movements to "act without delay to stop this practice."
He singled out "the increase in sexual violence against girls and women, particularly in Darfur," the vast western region where the government and various rebel groups have been fighting for more than three years.
"Grave violence against women in Darfur continues to worsen," he said. "Girls have been targeted in interethnic conflicts as a deliberate form of humiliation of a group, and as a means of ethnic cleansing."
Annan stressed the "urgent need for national authorities to rigorously investigate and prosecute responsible parties and to put into place measures for the protection of girls and women who are more vulnerable, especially among young internally displaced populations."
He said he was especially concerned about the "systematic abduction and kidnapping of children, particularly in Darfur," and urged the government and rebel groups to stop the practice.
The report, covering the period from May to July, said individual commanders of armed forces and groups in Sudan should "bear responsibility" for the systematic violations against children by their forces. But it said the Sudanese government and the government of southern Sudan are "directly accountable" for violations by individuals under their command.
The exact number of violations against children is difficult to track because of restricted access in certain parts of the country like eastern Sudan, and the lack of reporting by the government or the victims, Annan said.
The government signed a peace agreement in January 2005 ending a 21-year civil war with southern rebels. A May peace agreement signed by the government and one of the major rebel groups in Darfur was supposed to help end the three-year conflict there, but instead it has sparked months of fighting between rival rebel factions.
More than 200,000 people have been killed in Darfur since 2003 when ethnic African tribes revolted against the Arab-led Khartoum government.