In an appearance on "The Jay Leno Show" Thursday, the Republican said he challenged the Democratic president to a two-on-two showdown when they talked after his upset victory on election night.
Brown says he'd team up with his daughter, who plays guard at Boston College.
Brown, whose basketball skills helped him win college admission, predicted a tough match but added, "I think we'd have the upper hand."
The once little-known state legislator jolted the national political landscape by capturing the Senate seat long held by the late Democratic Sen. Edward Kennedy. He appeared briefly on Leno's show via satellite hookup from Massachusetts.
Prodded by Leno, he revealed he stole records from a store when he was 12 years old, but after being scolded by a judge saw the error of his ways.
"I haven't done anything like that since," Brown said.
He also named Sylvester Stallone's "Rocky" films among his favorites and croaked out a classic line: "Adrian ... I did it."