On his days off, neighbors said he tinkered with antique cars, practiced classical piano and tended to a garden, even building his daughter a flower arbor for her high school graduation.
But something went wrong on Wednesday, when Smith was behind the wheel of a ferry that slammed into a pier, killing at least 10 people and injuring more than 60.
"That's a terrible amount of grief to carry around that you might be responsible for 10 people dying," said Amy Crane, who lives across the street. "I feel so bad for him and his family."
Smith, 55, remained hospitalized in critical condition Thursday after police officials said he fled the scene of the crash and tried to commit suicide at home, slashing his wrists and shooting himself twice with a pellet gun.
Investigators confirmed Thursday that Smith was at the helm of the Andrew J. Barberi in July 1995 when the vessel crashed into the Staten Island dock. The accident was blamed on a mechanical problem after the propeller failed to switch into reverse. No fatalities were reported, although there were a number of injuries.
No one answered the door at Smith's brick home in Staten Island, surrounded by blue police barricades. His family, through an attorney, released a statement offering prayers for the victims and urging people not to rush to judgment.
Crane described Smith as "a standup guy. He's with his family all the time. Hardworking, jovial."
Smith lives with his wife, Lorraine, a teenage daughter and a "very, very young baby" daughter, Crane said. He also has two children from a previous marriage.
Neighbor Stella LoBianco said she often heard Smith practicing classical piano after work. "It was very beautiful," she said.
Smith made an archway festooned with flowers for one of his daughters after she graduated from high school, and at one time spent hours refurbishing a 1941 Ford sedan in his driveway.
LoBianco said that Smith was likely devastated by the horrific crash.
"He had to be totally traumatized when he left that boat," she said. "If he saw what had happened, he would just have to be totally traumatized by the effect of what happened."
"Even if it's not his fault, that's such a heavy thing to carry," Crane said. "For one person to have to take responsibility for all that is ridiculous."
Investigators were still waiting to interview Smith about the crash, but Staten Island Councilman Michael McMahon said Thursday he had been told Smith may have lost consciousness because of health problems. He said the officials were looking into whether Smith had failed to take blood pressure medication.
The city's Department of Transportation, which runs the ferry, could not immediately say if he suffered from a medical condition that might have played a role Wednesday.
Department spokesman Tom Cocola said Thursday that, as a ferry pilot, Smith would have undergone a routine physical and drug and alcohol tests before being certified by the Coast Guard. All crew members' records are being reviewed, he said.