Bill Clinton, George W. Bush and other former presidents who earn lucrative speaking fees and draw other income would no longer be able to count on taxpayer dollars to pay for their post-White House office space and staff under a bill in the House. 

On a voice vote, the House Oversight panel backed a measure Tuesday to limit taxpayer dollars for expenses, including travel, incurred by ex-presidents who earn more than $400,000 a year. 

U.S. taxpayers paid a total of $3.5 million last year in pensions and benefits to the four living former presidents, including $1.3 million for Bush and $950,000 for Clinton, according to a report by the Congressional Research Service. Most of that money was for sprawling office space in Dallas and New York, respectively. 

Both Clinton and Bush, like other ex-presidents before them, have earned millions in speaking fees since leaving office. 

The oversight committee acted just days after Hillary Rodham Clinton reported that she and her husband earned more than $30 million combined in speaking fees and book royalties since January 2014. The earnings put the couple in the top one-10th of 1 percent of all Americans. 

The House bill would set presidential pensions at $200,000 a year, nearly the same as the current amount, with an additional $200,000 set aside for office space and other expenses. The bill would reduce expense payments by $1 for every dollar above $400,000 earned by a former president. 

Under the legislation, ex-presidents who earn more than $600,000 a year would not receive federal funds for office expenses or travel. Presidential pensions would not be affected by the amount of income earned. 

Rep. Jason Chaffetz, R-Utah, chairman of the House Oversight panel and a co-sponsor of the bill, said it was not aimed at anyone, but was a simple matter of fairness. 

"History shows that former presidents do very well financially after they leave office," Chaffetz said in a statement before Tuesday's vote. "In fact all former presidents are millionaires, making it unlikely that they depend upon their taxpayer-funded allowance to make ends meet." 

Chaffetz, who has introduced similar bills in the past two sessions of Congress, missed Tuesday's vote because of an emergency operation to remove his gall bladder. 

Rep. Elijah Cummings of Maryland, the senior Democrat on the panel, co-sponsored the latest measure with Chaffetz. 

"Taxpayers should not have to pay for a former president's allowance if the former president is making a comfortable living earning more than $400,000 a year after leaving office," Cummings said. 

The report by the Congressional Research Service said taxpayers paid more than $420,000 last year for Bush's 8,237-square-feet office in Dallas. Clinton's 8,300-square-feet space in New York cost nearly $415,000. 

Taxpayers also shelled out nearly $180,000 for office space in Houston for George H.W. Bush and $109,000 for work space in Atlanta for Jimmy Carter. 

Carter, who left office in 1981, received a total of $470,000 in pensions and benefits last year, while George H.W. Bush received $837,000. He left office in 1989. 

The totals do not include money spent by the Secret Service to protect the former presidents and their families. The House bill would not affect Secret Service protection. 

Rep. Glenn Grothman, R-Wis., called the bill "a very good bill, a very necessary bill, given what we've seen going on here." 

Grothman said it was important to reduce federal payments to ex-presidents as the income of those "trading on their office" increased. 

"Hopefully it will restore some dignity to the office of ex-president," Grothman said. 

Hillary Clinton's presidential campaign reported Friday that the couple amassed more than $25 million in speaking fees and that Hillary Clinton earned more than $5 million from her 2014 memoir, "Hard Choices." 

Bush has earned at least $15 million for more than 140 paid speeches since he left office in 2009. 

The Former President's Act was passed in 1958 after Congress learned of financial difficulties faced by Harry Truman, who left office in 1953. Truman complained that it cost him $30,000 a year to reply to letters and requests for speeches, the report said.