However, people in the industry insist the sport is not just for the wealthy. In a down economy, it's not just humans who are feeling the pinch when it comes to retirement.
With an average racing career lasting only two to five years, each year thousands of horses cross the finish line for the last time.
There is no federal legislation concerning the fate of horses when their racing days are over and the industry has been slow to self-regulate, leaving much of care to a patchwork of charitable organizations.
"Most farms, you know they race and they enjoy these horses and they retire them, but there are horses that are left behind," says Tom Ludt, chairman of the Thoroughbred Retirement Foundation.
The Thoroughbred Retirement Foundation, or TRF, accepts horses when their racing days are over and their owners aren't able to, or don't want to, take care of them. The organization places the animals at satellite farms throughout the country with the goal of retraining them and finding them good homes. It is the largest organization of its kind in the United States.
"I just think there are people who get overextended that are outside of their means, and like anything… you just have to sell it," says Ludt of why owners would give up their horses.
The TRF is funded entirely by donations, and Ludt says the volatility of the U.S. economy has severely impacted their funding stream. Despite reports the organization has allowed funding issues to affect the care of some horses, Ludt is emphatic that no matter what the money situation, all animals they take on are well maintained.
Rumors of poor care of retired racers in a down economy inevitably lead to the question of why the obligation falls to charities in the first place. Industry insiders say many owners just don't have the means to keep their horses and the perception of racing as the "sport of kings" is just not true.
David Brown, President of the Finger Lakes Horsemen's Benevolent and Protective Association in New York says, "If you want to go to Dubai, it's the sport of kings. If you want to come to the Finger Lakes it's the sport of people who love animals."
Brown is associated with an innovative program at the Finger Lakes Casino & Racetrack in Farmington, NY. The track has a thoroughbred adoption barn on site, and $4 from every start at the track helps fund racehorse retirement.
"It's a pioneer type of program and all the other money that's raised for the program comes through private donation," says Steven Martin, spokesman for the Finger Lakes Casino & Racetrack.
Brian Moore is the President of the Board of the on-site thoroughbred adoption program. "We proclaim to be the first where this has happened in the country," he says, " but we certainly don't want to be the last and we hope other people follow our lead."
Owners who have horses at the upstate New York track applaud the program and say most owners try to do as much as they can to look after retired horses, but it is expensive.
Owner and trainer John Tebbutt's love for his horses is clear, but explains that, for many people, keeping horses when they can't race is just out of reach.
"Horses race until a lot of times… maybe six years old, and a horse's lifespan goes to 20 to 25 years old. It costs $50,000, maybe $100,000 to carry them those next 20 years and that's cost prohibitive for most people.
Fellow owner and trainer Phyllis Shetron says, "The average horse owner does care what happen(s) to their horses… Some people just don't know what to do with their horses in the end."
Industry wide, nearly everyone agrees more has to be done to prevent the neglect of retiring races horses. Economic conditions have made the job especially difficult.
"There are few dollars out there, there's no question," notes Alex Waldrop, president of the National Thoroughbred Racing Association. "But we have to remain vigilant." He continues, "We can't let the economy become an excuse for not taking care of the horses."
The National Thoroughbred Racing Association has an accreditation process through its Safety and Integrity Alliance. The process has been in place for less than three years and already 20 of the roughly 70 tracks nationwide are accredited. In order to receive accreditation, a track must have an established relationship with some sort or racehorse retirement program.
Waldrop says the industry is taking notice, and change is happening.
"We're doing better than ever," he says, "We could do more, there's no question there's more we could do. But I think the industry is aware of the fact that it has to step up and take care of our retired athletes."
What sort of change, remains to be seen. Ludt recommends an industry wide structure whereby all members of the industry contribute.
"I think the racetracks, I think the state associations, the TRF, the horsemen’s associations, they're all realizing this is a situation… that needs to be structured and organized."