BILOXI, MISS. -- While the struggling economy is at the top of the agenda for the governors convening this weekend at their annual meeting, most attending the National Governors Association meeting are not rallying behind calls for a second stimulus package from Washington, D.C.

"I'm worried about the deficit," Gov. Bill Richardson, D-N.M., said Sunday. "You got to be careful about another stimulus right now because I think the first stimulus package is working and we should let it run its course."

Gov. Brian Schweitzer, D-Mont., said that calls for a second wave of federal money are "premature" and unneeded.

The federal government approved the $787 billion spending and tax plan in February. Only a fraction of the cash has been disbursed so far, primarily to the states.

Gov. Haley Barbour, R-Miss., chairman of the Republican Governors Association, was initially against the first stimulus package passed by Congress in February. But Barbour now speaks favorably of the legislation that poured millions of federal dollars into his state. 

However, Barbour said a second stimulus package is a "bad idea."

"The American people right now are rightfully concerned about this incredibly unbelievable rate of spending," Barbour said. "I'm one of those Americans who is worried about the effect this is going to have on the value of the dollar."

The governors are attending the 101st National Governors Association conference, but it was debatable how much work they would be able to accomplish this weekend because fewer than half of the nation's 50 governors are in attendance.

FOX News' Jonathan Serrie and David Lewkowict contributed to this report.