The storm made landfall on Vietnam's northern coast Saturday night. Six fishermen remained missing after their boat sank as they sought shelter at the Paracel islands off Vietnam's central coast, said disaster official Pham Thi Kim Tuyen in central Quang Ngai province.
Tuyen said naval ships and fishing boats were searching for those missing.
Before making its way to Vietnam as a tropical storm, the typhoon slammed the Philippines and China last week.
The bodies of three missing Filipino fishermen were recovered, raising the country's death toll to 68 with 84 missing, the Philippines' National Disaster Coordinating Council reported Sunday. Conson then grazed China, killing two there.
In Vietnam's northern province of Quang Ninh, a man was reported missing after his small boat sank in the picturesque tourist attraction of Ha Long Bay, said disaster official Nguyen Tri Thai. More than 100 vessels were ordered to anchor before the storm struck.
A female Vietnamese tourist was missing after she was swept away by high waves while swimming, said another disaster official, Le Duy Thanh of Thanh Hoa province.
Provincial authorities called off plans to evacuate 137,000 people because the weakened storm was not as severe as expected, said Thanh.
The national floods and storms control department said Sunday nearly 20,000 people had been evacuated from four coastal provinces. Some 300 homes were damaged and 27 boats destroyed.
Meteorologists said Conson was now a tropical depression and will fade away later Sunday.
Rain was forecast for some northern Vietnamese provinces Sunday, though it was unlikely to cause major flash floods and landslides that authorities initially feared.