"For him, he said, the World Cup is finished," Toure said.
The 32-year-old Drogba was taken off 15 minutes into Ivory Coast's 2-0 win over Japan after a high challenge by Japan defender Marcus Tulio Tanaka. He was then rushed to a hospital.
"We can just see on his face that he was sad, and when he is like that you can't ask more," Toure said. "It is difficult for us because he is such a really important player."
Team spokesman Eric Kacou told The Associated Press that "it is a fracture in his right arm," but coach Sven-Goran Eriksson said the team hadn't officially ruled out its captain.
"We are not sure yet," Eriksson said, saying Drogba's participation was "still a possibility."
Toure said Ivory Coast's players were hoping for a surprising recovery, and said the team would know more later Friday.
Tulio said he had no intention of hurting Drogba.
"God and everybody in the stadium saw it was a normal challenge," said the Brazil-born defender, who was shown a yellow card. "I do hope he gets better in time to play in the World Cup."
Drogba scored 29 goals in 32 Premier League matches this season, helping Chelsea win the title.
Friday's friendly was the final warmup for Ivory Coast and Japan before they travel to South Africa. The Ivorians open the World Cup on June 15 against Portugal in Port Elizabeth, while Japan begins a day earlier against Cameroon in Bloemfontein.
Drogba appeared to be holding his right side after Tulio's challenge. After returning from the hospital, the striker entered the team bus without speaking to reporters.
Eriksson said the injury was unlucky, and didn't blame Tulio.
"It wasn't bad or stupid. He didn't want to hurt anyone," the Swedish coach said. "Football is a game of contact."
Associated Press Writer Bradley S. Klapper in Geneva contributed to this report.