While Cowell said negotiations with Abdul haven't begun yet, he told Deadline.com "there's a good chance it will be her."
"I missed her the second she left [Idol]," Cowell said. "Always loved working with her even though she can be a pain. ... I don't know what it is about her, but I've always clicked with her. You just have to get that chemistry, and she's right. I've never found anyone better than her."
Abdul would join Cowell and previously announced judge Antonio "LA" Reid. Cowell also confirmed reports that British singer Cheryl Cole will likely be the fourth judge at the table. Although Cole, who also is a judge on the British version of 'X Factor,' is relatively unknown to U.S. audiences, Cowell defended the choice, saying Cole is "special."
"Shes a great judge. She's smart," Cowell said. "It's just a hunch. If people take to her like the British public did, I think she's going to do really well in America. And Fox was desperate to hire her."
As for the other rumored judges? Cowell revealed that he's never spoken to Snoop Dogg about the job. He met with Cee Lo, but never offered him the job, and said he would have loved to have Mariah Carey, but she was not available due to her pregnancy.
The 'X Factor,' which premieres this fall, will search the nation for undiscovered artist or group. The winning act will receive a $5 million recording contract with Syco/Sony Music.
Fox did not immediately respond to TVGuide.com's request for comment.
Are you happy to see Paula and Simon reunited?
Other Links From TVGuide.com