WASHINGTON -- Talks between CIA Director Leon Panetta and his Pakistani counterpart did not include an "ultimatum session" for the United States to limit its operations inside the terrorist-laden Afghan neighbor, U.S. officials said after a closed-door meeting Monday in Washington, D.C.

The exchange between Panetta and Gen. Ahmed Shuja Pasha, the head of Pakistan's Inter-Services Intelligence, or ISI, was the latest discussion between the two sides since CIA operative Raymond Davis was quietly removed from Pakistani custody after Davis shot and killed two Pakistanis in Lahore, setting off demonstrations in the country. Davis said he acted to fend off two gunmen trying to rob him. The families of the two deceased received a financial settlement.

Reports have suggested the Pakistanis came to town requesting more than 300 CIA operatives leave the country immediately and U.S. Predator drone strikes be halted, or at least restricted to a narrow part of North Waziristan, where they were initially carried out before President Obama and his national security team began expanding the program.

Last year, more than 119 drone strikes were carried out in Pakistan, but there hasn't been one since March 17.

Unconfirmed reports also said Pakistan wanted to eject 120 Special Ops forces who are training the Frontier Corps, but it is not clear whether the carefully timed leaks from Pakistani officials may have been a negotiating ploy to regain leverage in the relationship.

A U.S. official told Fox News that the Pakistanis are out "in a very public way" to make their case and appear to be using the Davis episode "as leverage."

The Pakistanis never seem to lose an opportunity "to take advantage of a crisis," the official said.

But the source, familiar with discussions between the two, said that while the Pakistanis have voiced a range of concerns in recent weeks, what they actually want depends on which Pakistani official is asked.

A CIA spokesman said the luncheon was an opportunity for the two sides to sit down and work on issues vital to counterterrorism operations.

"Today's exchange emphasized the need to continue to work closely together, including on our common fight against terrorist networks that threaten both countries," George Little said after the meeting Monday.

A U.S. official told Fox News that the meeting between Panetta and Pasha included a frank discussion, but the two leaders discussed common interests with a few concerns, "all of which can be sorted out."

"Panetta -- who made it clear that his first priority is protecting the American people -- and Pasha had a conversation that reflected a sense of partnership and desire to move forward.  This wasn't some kind of ultimatum session, as some press reports have suggested it might be," the official said.

"The Pakistanis have asked for more visibility into some things, and that request is being talked about -- along with a host of other topics, including ways to further expand the partnership. The bottom line is that joint cooperation is essential to the security of the two nations. The stakes are too high," the official added.

U.S. intelligence and military officials believe factions in the Pakistani intelligence agency support Taliban and other militant groups, which are killing U.S. troops just across the border in Afghanistan.

Former CIA officials expressed skepticism that the ISI could be trusted to fight terrorism on its own without the current level of CIA staffing on the ground, citing the ISI's alliance with the Haqqani terror organization and the rise of the Taliban after the U.S. walked away from Afghanistan in the wake of the Soviet Union's defeat.

But Pakistani and U.S. officials confirm that a CIA tip led to Pakistan's capture this year of Indonesian Umar Patek, one of the accused masterminds behind the 2005 Bali, Indonesia, bombing. And U.S. officials add that some joint missions have been carried out despite the recent diplomatic impasse.

White House spokesman Jay Carney on Tuesday said the relationship between Pakistan and the United States is important and the two sides have a shared goal of defeating insurgents. However, he would not discuss specifics of the cooperation.

State Department spokesman Mark Toner acknowledged that relations with Pakistan are going through a difficult time, but "it's not a one-dimensional relationship" and the two sides are rebuilding it.

Asked about the Pakistani efforts to limit U.S. counterterrorism efforts, Senate Foreign Relations Committee Chairman John Kerry, D-Mass., said he understands why some reduction in U.S. effort in Pakistan has occurred.

"If you have got a bunch of operatives who are running around your country, if it had happened here in the United States, someone was shot dead in the center of Washington by one of those operatives, there'd be a cry in our country, too. I think we need to be thoughtful about that. They're insisting on knowing who is there and doing what and I think we can work through these kinds of things," Kerry said.

Kerry added that their government can do more to help "decharge" the atmosphere after incidents like the one involving Davis and Joint Chiefs of Staff Chairman Adm. Mike Mullen held meetings with Pasha on Tuesday to discuss those efforts.

Sen. John McCain, R-Ariz., who was appearing with Kerry at a news conference on Internet privacy, added that he believes the Pakistanis "are becoming more and more convinced we are leaving Afghanistan," and since they remain in the region after the U.S. is gone, is covering all their bets.

Fox News' Jennifer Griffin, Catherine Herridge and Trish Turner and The Associated Press contributed to this report.