Libertarian voters, advocating everything from less government to legalized pot, seem the perfect wildcard in any November election, peeling off Democratic or Republican votes.

However, their stronger allegiance to the Republican Party and frequent opposition to establishment candidates continue to create more problems for GOP nominees in general elections. And this year may be no different, as libertarian candidates threaten to drain votes from Republicans in key races.

The most high-impact race this year will likely be in New Hampshire, where Libertarian candidate Gardner Goldsmith is complicating Senate Republican candidate Scott Brown’s bid to defeat incumbent Democratic Sen. Jeanne Shaheen and help his party take control of the upper chamber.

Brown trails Shaheen by 10 percentage points (39-49) with Goldsmith taking 2 points and 9 percent of voters still undecided, according to the most recent Suffolk University/ Boston Herald poll.

While Goldsmith’s 2 percentage points might appear too small to decide the race, political observers argue New Hampshire voters have a history of supporting independent candidates, which means that Goldsmith could take a big chuck of the undecided vote and block Brown’s potential path to victory.

In addition, even if Brown wins all 9 percent of the undecided vote, Goldsmith still holds the deciding 2 percent, says David Paleologos, director of the university’s Political Research Center. He also points out that 2 percent of registered Republican voters in the June 20 poll favored Goldsmith, compared to zero for Democrats.

Republicans need to win a net six seats this fall to take control of the Senate.

In North Carolina, Libertarian candidate Sean Haugh is also complicating Republicans’ plans to defeat Sen. Kay Hagan, considered one of the most vulnerable Democratic incumbent senators in 2014.

A July 22 poll by the Democrat-leaning Public Policy Polling shows Hagan leading Tillis 41-to-34 percent, with Haugh taking 8 percent of the vote and 16 percent of voters still undecided.

Beyond the percentage of undecided voters being large enough to decide the race, Haugh’s 8 percent could become an even bigger factor should he fail to reach 10 percent and qualify for the senatorial debates starting in September, and as a result of that fade.

The same poll also shows that 42 percent of his supporters would instead likely back Hagan and 39 percent would likely break for Tillis. However, whether Hagan or Tillis would benefit the most is largely unclear because 16 percent of Haugh voters were not sure about an alternative choice.

The Libertarian agenda, particularly its less-government message, still appears to be a driving force in the Republican Party.

Libertarian-minded Kentucky Sen. Rand Paul, for example, is ahead in many early GOP 2016 presidential polls. But the Libertarian Party is quick to extinguish the spoiler theory and argues it has a foot in both major camps.

“Gay marriage, less taxes, there’s overlap there,” Libertarian Party President Wes Benedict told FoxNews.com earlier this week. “From a marketing perspective, we want to distinguish ourselves from Democrats and Republicans. But we do want to attract voters from both sides.”

Benedict points to exit polls of the 2013 Virginia gubernatorial race that appear to upend the theory that Libertarian Party nominee Robert Sarvis cost Republican nominee Ken Cuccinelli the race against Democratic nominee Terry McAuliffe, in which he lost 45.5-to-48 percent with Sarvis taking 6.6 percent of the vote.

The exit poll by Edison Media Research shows that 5 percent of Sarvis voters would not have cast a ballot had he not been in the race, 1 percent would have gone to Cuccinelli, moving him to 46.5 percent, and McAuliffe would have stayed at 48 percent.