The ruling, on a suit filed by relatives of Japanese, Chinese and South Korean war dead, is the first by the nation's top court on a case alleging the visits violate the division between religion and the state.
The Supreme Court bench, presided over by chief judge Isao Imai, threw the case out Friday, court official Takashi Ando said. He declined to give other details, but said the court fees must be paid by the plaintiffs.
The suit had targeted Koizumi's first visit to the shrine as prime minister, made on Aug. 13, 2001.
The Osaka High Court also threw out the suit in July 2005, saying there was no legal foundation for the plaintiffs' claims for damages. The court, however, did not touch on the constitutionality of the visits.
Koizumi has gone to Yasukuni Shrine five times since taking office in April 2001.
His visits have outraged China and South Korea, which consider the shrine a glorification of Japan's militarist policies in the first half of the 20th century.
Koizumi says the visits are made in a private capacity to pray for peace, but opponents have filed a series of lawsuits in Japan arguing that they violate the Constitution's division of state and religion.
Japanese courts for the most part have rejected such suits demanding compensation, though they often sidestep the issue of constitutionality. In one exception, the same Osaka court in October 2005 found Koizumi's visits were unconstitutional.
That ruling concurred with a decision by the Fukuoka District Court in April 2004. Neither court, however, approved the plaintiffs' demands for compensation.
Before Friday's ruling, plaintiffs were already pessimistic about the outcome.
"I don't expect the Supreme Court to reverse the High Court decision," Masaharu Hishiki, one of the plaintiffs, said last week. "I also think it's unlikely that the Supreme Court will make a judgment on the constitutionality of the visit."