The future is gnawing at homeowner Scott Klimek. Will the interest rate on his adjustable-rate mortgage go up? Will he be able to afford the monthly payments if it does?

Klimek, 47, bought his home in Brighton, Colo., with a five-year ARM. "If it was, like, this year that it was going to adjust, then I would really be worried, but I'm just moderately worried," says Klimek. He has four more years until the rate on his mortgage changes.

He is not alone in his concern.

Click here to visit FOXBusiness.com's Real Estate Center.

An AP-AOL Real Estate Poll found that more than one-third — 36 percent — of those surveyed with adjustable-rate mortgages worry that they won't be able to afford their monthly mortgage payments if their interest rates increase.

Sixty-four percent, meanwhile, aren't sweating it and say they're not concerned about such a scenario.

The poll also found that those who believe that housing prices will go up were more likely to have adjustable-rate mortgages than those who think prices will go down or stay the same.

In the boom days, home sales racked up record highs for five years in a row, with the last one in 2005. Home values skyrocketed, growing by double digits, which made homeowners feel wealthy but some house hunters feel poor. Still, low mortgage rates — at their best levels in decades — lured buyers, including some people who stretched to buy a pricey home that they otherwise probably couldn't afford.

The situation is different these days. Mortgage rates have gone up. Home prices have dropped in some markets, while in others they are not rising anywhere near the pace they had during the boom times.

This new climate has clobbered some homeowners.

Joann Clapp, 50, of Cynthiana, Ky., has suffered a double whammy. She recently moved and hasn't been able to sell her old home, so she is paying on two adjustable-rate mortgages. The rise in interest rates has hurt her finances "big time," she says.

Of all U.S. homeowners with a mortgage, 24 percent have an adjustable rate, while 76 percent have a fixed-rate mortgage, according to the Mortgage Bankers Association.

Initial rates on adjustable mortgages have gone up faster than those on longer-term, fixed-rate loans.

Last week rates on one-year ARMs stood at 5.60 percent, up from 4.91 percent a year ago, says Freddie Mac. Rates on 30-year mortgages averaged 6.40 percent, compared with 6.15 percent a year earlier.

Greg McBride, senior financial analyst for Bankrate.com, says the sharp rise in short-term interest rates can mean a big jolt to people with adjustable-rate mortgages. "Someone with a $200,000 loan taken three years ago who is now facing an initial rate reset can see the payment jump by nearly $400 per month," he says.

Klimek said he was advised that an adjustable-rate mortgage was his best option to finance the home he wanted to buy. "It just worked out to get us in the house," he recalls. "We were in a townhome and we wanted to get out of there."

He would like to refinance and get the security of a fixed-rate mortgage before his ARM adjusts in four years but isn't sure he'll be able to swing it. "I think it could happen," he says.

McBride's advice for people now facing a rate change on their ARM: refinance into a fixed-rate mortgage. "Even though you will see a higher payment, you bite the bullet once. The payment doesn't increase nearly as much as if you sit back and do nothing and only face the music of that rate reset later," McBride says.

The AP-AOL poll found that a third — 35 percent — of likely future buyers say they'll seek an adjustable-rate mortgage to finance their home purchase.

People most likely to seek adjustable-rate mortgages are first-time home buyers, younger people, those with less education and lower incomes, unmarried adults and minorities, the AP-AOL poll says.

Experts suggest that such prospective home buyers do some financial soul-searching before taking out an adjustable-rate mortgage.

"They should not assume that the value of their home will rise consistently or even considerably, and they should not assume that their incomes will rise rapidly," says Lynn Reaser, chief economist at Bank of America's Investment Strategies Group. "They should examine the worst-case scenario ... whether or not they could afford their payments with their current home price and income level."

And, first-time buyers shouldn't get locked in to the notion that their first home must be their dream home.

"Many Americans have lost the view of the so-called starter house," says Reaser. "People may want to take a more conservative view of that first home before they move to the mega-mansion."

The AP-AOL Real Estate poll of 2,001 adults, including 289 recent homebuyers and 401 likely future homebuyers, was conducted by telephone Sept. 19-26 by Ipsos. The poll had a margin of error of plus or minus 2 percentage points for all adults, 6 percentage points for recent homebuyers and 5 percentage points for likely future homebuyers.

Click here to visit FOXBusiness.com's Real Estate Center.