Red-faced Portuguese Prime Minister Pedro Passos Coelho has admitted he is "not perfect" after it was revealed he failed to pay his social security contributions during five years, before he came to office.

Passos Coelho's center-right coalition government launched a crackdown on tax evasion as part of an austerity program when it took power in 2011, following a 78 billion euro ($87 billion) bailout that spared Portugal from bankruptcy.

A newspaper revealed last weekend that he failed to pay almost 4,000 euros in social security from 1999-2004. The prime minister said he wasn't aware of the missing payments and paid the outstanding sum after the press report appeared.

Passos Coelho told his Social Democratic Party on Tuesday, "I have my imperfections."

He says he won't resign over the controversy.