The final truckloads of rubble had just left the World Trade Center site in the spring of 2002 when Mayor Michael Bloomberg foretold the difficulties ahead for the process of building a memorial to the victims of the terror attack.

The former CEO, in public office barely six months, told a group of business leaders he couldn't envision the end result.

"I can tell you the process, however," Bloomberg said. "The process is everybody yelling and screaming for a number of years and then somebody taking charge and just doing it."

He was remarkably right. Amid all the shouting — some directed at him — the mayor himself is taking over the memorial's multimillion-dollar foundation, where board members hope his philanthropic credentials and star power will breathe life into lethargic fundraising.

That surprised many people because of his awkward history with the memorial and victims' families.

His predecessor, Rudolph Giuliani, had suggested when he left office that the entire site should be made into a "soaring, monumental" memorial.

Bloomberg, however, was focused on stabilizing the city's shaky post-9/11 economy, and the pragmatic billionaire could not support giving up 16 acres of prime downtown Manhattan real estate. He came across as business-oriented and harsh.

Victims' relatives were angered when he said downtown residents wouldn't want to live next to a "cemetery." Many were already unhappy because at the start of his term Bloomberg did not attend every firefighter's funeral as Giuliani had done, although he quickly changed his ways.

The next year, he took heat for comments reportedly made during a meeting with Diane and Kurt Horning, who lost their son in the attack and were upset that the city buried the sifted trade center dust in a landfill. They say that dust still contains specks of human remains, and are suing the city in federal court to force its removal.

Diane Horning says Bloomberg should not head the emotionally charged memorial effort because he was "dismissive and abrupt" about their views on grieving and remains.

Bloomberg's opponents also say he lacks compassion for rescue workers sickened by toxic trade center debris and has ignored demands to group the names of victims on the memorial by where they died and who employed them, rather than listing the names randomly.

"I don't get any warmth from the man — I don't feel that he can understand the need for memorializing the victims or the feelings of the families," Horning said. "Does he understand the financials? Of course he does, but that's not enough."

Christy Ferer, who lost her husband on Sept. 11 and serves as Bloomberg's liaison to the families, says the public dramatically misunderstands him.

Ferer said he cares deeply about the families and has been instrumental in countless projects on their behalf.

"He is an amazing, philanthropic guy, but that doesn't mean he's warm and fuzzy, and sometimes people can potentially misread that," she said. "Maybe these families can't recognize his ability to execute his vision in the long run, and therefore think that he's all about lack of feeling."

The mayor lost friends on 9/11 and endured his own grief early in life, but rarely mentions either. His father, who had a weak heart, died during Bloomberg's junior year in college.

In his autobiography, Bloomberg noted that "today, he could have survived; medicine wasn't as capable then as it is now."

It is a revealing passage: The mayor, who wears his father's watch as a remembrance, donates hundreds of millions to medical research.

Many say this is how he deals with adversity, by focusing on what can be changed rather than dwelling on the past.

"His energy is totally directed forward, and sometimes his way of grieving is really not to look backwards," Ferer said.

He has also given $10 million to the memorial foundation. As its chairman, the Republican mayor will likely tap his business connections and wealthy friends for more donations to the project, which is expected to cost more than $700 million. Nearly $145 million has been raised. Construction on the memorial has begun and officials hope to finish in 2009 — Bloomberg's last year in office.

"What we're really seeing is that he understands how central this is to the nation, to the city and to the families," said Mitchell Moss, a New York University urban planning professor. "Even though he has not been that close with them, he understands the power of this and the significance that it has."