They have become fed up with the violence, the bureaucracy, the political finger-pointing, the sluggish rebuilding and the doubts about the safety of the levees.
"The mayor says, 'Come back home. Every area should come back.' For what?" said Genevieve Bellow, who rebuilt her home in heavily damaged eastern New Orleans but has been unable to get anything done about the trash and abandoned apartment buildings in her neighborhood and may leave town. "I have no confidence in anything or anybody."
A survey released in November found that 32 percent of city residents polled may leave within two years. University of New Orleans political scientist Susan Howell, who did the survey, said more will give up if the recovery does not pick up speed.
In fact, figures from the nation's top three moving companies suggest more people left the area than moved into it last year.
"People are in a state of limbo. They're asking, 'Is it worth it for me to stay? Is it worth it to invest?' If you don't feel safe, from crime or the levees, and you see destruction every day when you drive, it becomes discouraging," Howell said.
If there is an exodus, it could mean more than just a shrunken New Orleans. It could mean a poorer city, financially and culturally, and a more desperate one, too, since the people likely to leave are the most highly educated and younger.
Mayor Ray Nagin and Gov. Kathleen Blanco have urged residents to return under rebuilding plans with names like Bring New Orleans Back and Road Home. The mayor has warned that the recovery will take a decade and has urged people not to give up hope.
But New Orleans' population appears to have plateaued at about half the pre-storm level of 455,000, well short of Nagin's prediction of 300,000 by the end of 2006. And in many ways, it is a meaner city than it was before the hurricane.
New Orleans ended 2006 with 161 homicides, for a murder rate higher than it was before Katrina and more than 4 1/2 times the national average for cities its size. After starting 2007 with practically one killing a day, the city has at least 19 slayings so far this year.
The criminal justice system is in disarray, with public defenders so overworked and witnesses so reluctant to testify that the courts are revolving doors, putting criminals back on the street. Mistrust between police and the public is running high, in part because seven officers were arrested in a deadly shooting during the chaotic aftermath of Katrina.
Nagin and Police Chief Warren Riley announced a plan last month to crack down on crime with checkpoints and the putting of more police on the beat.
For Jennifer Johansen, it is too little, too late. Johansen's neat yellow house in New Orleans Irish Channel is for sale, and the nurse, who returned to the city after Katrina, hopes to be in Seattle before spring.
The gunfire she used to hear until about a month ago made her uneasy about watching TV in her living room, and she yearns to live in a vibrant, safe city.
"I kept thinking, things would get better. But it just took too long for a response from the city, the mayor, the police chief, to address the increased crime," she said.
Louisiana demographer Elliott Stonecipher said: "You get the sense talking to people on the ground in New Orleans that a lot of people are right on the edge. They're just about to the point where they believe they have to decide."
Blanco's Road Home program, born 10 months after the storm, has been vilified by politicians and civic leaders as too slow to distribute $7.5 billion in federal aid to buy out homeowners or help them rebuild. As of Feb. 5, Road Home had taken 105,739 applications and resolved only 532 cases, granting $33.8 million. At the current rate, Road Home would take more than 13 years to complete.
Sen. David Vitter, R-La., called Road Home a debacle. In hopes of jump-starting the neighborhood rebuildings, the mayor has put in place a gap-loan program to let homeowners borrow on their promised Road Home grants.
City, state and federal officials have traded the blame over the slow distribution of relief aid.
So far, the federal government has earmarked about $750 million for infrastructure projects. The state homeland security department, charged with distributing the money, has given out only about half that. The governor said the city has been slow to complete the paperwork.
It was that kind of back-and-forth that prompted Ken White and his wife, Kathy, to give up and move to New York last year.
"We came back a month after the flood and thought about what we could do to stay and rebuild, but it became apparent to us it would take a long time and be very difficult," said White, who was director of emergency psychiatry at Charity Hospital when Katrina hit. "We were appalled by the ineptitude of government on all levels."
Gregory Hamilton, a longtime resident of eastern New Orleans, said he plans to stay, but is frustrated, too. "Everybody wants to follow the recovery. Nobody wants to lead the recovery," he said.
Some frustrations are rooted in the persistent widespread damage as well as the lack of a comprehensive rebuilding plan.
On many streets, newly rebuilt houses stand amid empty, decaying ones. In many neighborhoods, there are still heaps of smelly debris and FEMA trailers in front yards.
"Literally, if you want an aspirin in those neighborhoods, you have to go across the parish line or to an unflooded area," said Al Palumbo, a real estate agent.
A $14 billion rebuilding proposal is making its way through city government, and Nagin has appointed a recovery czar, Ed Blakely. But there is no timetable for implementation of a master plan, and no assurances the money will be there for it.
Blakely said he believes it will cost at least three times the $14 billion estimate to restore the city.
Brian Nolan, a photographer who moved to South Carolina after the city's failed levees left his home in Lakeview under 11 feet of water, said he did not believe the U.S. Army Corps of Engineers' assurances that floodwalls have been improved.
"After the storm, we were all pumped up to build a new house, but we lost that dream," he said.
Blanco, on a lobbying trip to Washington, said Thursday that she has received commitments from Democratic leaders that the recovery of the Gulf Coast will be a "front-burner" issue. Blanco also said that she, the mayor and several parish leaders have agreed to work together to break the "bureaucratic nightmare."
Demographer Greg Rigamer said that pressure on Road Home and the appointment of a recovery czar are positive steps, but that city must do more to rebuild schools, its health care system and housing to keep people here and bring others back.
"With every passing month," said UNO sociologist Rachel E. Luft, "it's less likely people will come back."