Brady, who led the New England Patriots to three Super Bowl championships, is demanding Yahoo stop running the ads and is seeking unspecified damages, according to a lawsuit filed in federal court in Los Angeles.
A spokeswoman for Sunnyvale, Calif.-based Yahoo declined comment, saying it was company policy not to discuss pending litigation.
Brady's complaint says that he was prominently featured in ads in Sports Illustrated magazine as well as on the Yahoo Web site without the athlete's permission.
Fantasy football is a game where sports fans assemble teams of real players from different NFL teams, competing against colleagues and friends in imaginary leagues. Their games are scored each week based on the individual performance of each of those players, with cash prizes often going to the winners.
In the lawsuit, Brady said the violation of his rights of publicity "was intentional, deliberate, willful and malicious."
Yahoo, which runs one of many fantasy football sites, offers a basic service at no cost, but charges up to $125 per season for access to advanced analysis tools and data.
The suit, which was filed November 20, says that Yahoo's fantasy football Web site generates more than 800 million page views a day during the football season and is part of a $4 billion a year industry that attracts some 8 million consumers.