The Tea Party doesn't want to be a one-election pony, and is starting early to make sure candidates heed their call for budget cuts and limited government ahead of the 2012 election. 

One faction of that movement, the Tea Party Patriots, held a small but vocal rally Thursday on Capitol Hill to pressure Congress to hold down spending for the rest of the year. The rally of about 100 people was a rather mild reminder of the Tea Party's might. Elsewhere, the strategists behind the multi-headed movement are plotting once again to make waves in the primaries and general election. 

With the budget debate in full swing, they warn that any Republicans failing to live up to their vows to cut spending could face a challenge in the primary season. But as in 2010, different groups have wildly different goals. 

Tea Party Nation founder Judson Phillips has been threatening to put somebody up against House Speaker John Boehner, griping that he's not done enough to get control of the debt. 

But over at Tea Party Express, Boehner is not considered a problem. 

Sal Russo, chief strategist with the group, said Tea Party Express so far has four targets, two from each party -- Democratic Sens. Ben Nelson in Nebraska and Debbie Stabenow in Michigan, and Republican Sens. Richard Lugar in Indiana and Olympia Snowe in Maine, both moderates. 

He said the group has been meeting with possible Senate challengers in those races since the beginning of the year, and has made some decisions about endorsements, though he declined to reveal them. 

He said that during the primaries, the group will look to galvanize fundraising for favored candidates and get supporters across the country to work the phones in key races. He said Tea Party Express would also run ads in the general election, with the ultimate goal of winning for Republicans a 60-seat, filibuster-proof majority -- something Democrats are determined to not let happen. Russo said the organization is waiting for redistricting before it starts strategizing at the House level, since districts could change, in turn affecting the vulnerability or strength of candidates. 

"Somebody may be vulnerable today, but when the map gets drawn, they may be in a safe district," Russo said. 

New for the Tea Party this cycle will be the chance to have an impact on the presidential race. Tea Party Express plans to invite presidential candidates to speak at stops on its planned bus tour later this year, and will co-host a presidential primary debate in the fall. 

The Tea Party already is part of the presidential primary trail. 

The Boston Herald reported that former Minnesota Gov. Tim Pawlenty, who recently launched his exploratory committee, plans to speak to the Boston Tea Party during its April 15 Tax Day rally. 

While candidates and Tea Partiers work through the awkward getting-to-know-you phase, Tea Party leaders are sending out a stern warning to incumbents. 

"If they don't cut the spending that they promised to cut, we're going to get back in 2012 and we are going to get them out of there," Ana Puig, with the Pennsylvania-based Kitchen Table Patriots, told Fox News. 

The Tea Party Patriots' Jenny Beth Martin also told Fox News that some members will "probably be primaried" if they don't live up to expectations. 

"We're going to have an impact in this debate, we're going to have an impact when it comes to the budget in 2012 and we're also going to have an impact in the debt ceiling and next year's elections," she said. "We're not going away. We're here. We're going to be vigilant." 

As Democrats openly spoil for a fight with the Tea Party by labeling their views "extreme," Republicans on the Hill continue to pay deference to the movement, in a sign of its importance in 2012. 

Boehner, who blames Democrats for the lack of progress on the budget, said lawmakers should "welcome" the input from people like those protesting on Capitol Hill Thursday. "I'm glad that they're engaged in the process," he said. 

Sen. Marco Rubio, R-Fla., a freshman senator elected with help from Tea Partiers in his state, warned lawmakers not to push their buttons. 

"A lot of these people have found their expression in the Tea Party movement. I think that's a positive development in American politics, and I think our leaders should not be criticizing everyday Americans who decide to express themselves through the political process," Rubio told Fox News. 

Recent polls have suggested enthusiasm for the Tea Party might be waning, at least in the drought period between elections. A CNN/Opinion Research poll out Wednesday showed about half of those surveyed having an unfavorable view of the movement, apparently the first time that number has been as high as it is for Democrats and Republicans. 

But Russo said he expects the movement to "grow" as the campaigns get underway. 

"I think we're in that frustration period among those that think the 2010 elections were determinative," he said. "We got one-half of one-third of the government. It was a first step." 

FoxNews.com's Judson Berger and Fox News' John Brandt contributed to this report.