Senior Pakistani leaders involved in the talks Friday include Gen. Ashfaq Parvez Kayani, head of the Army, and Gen. Ahmed Shuja Pasha, head of Pakistan's intelligence agency ISI.
According to a U.S. official, Panetta is in the country to highlight to the Pakistanis that the U.S. will continue to work with them. The official also said Panetta is there in his capacity as CIA director -- not as the president's nominee for defense secretary.
Panetta came before the Senate for his confirmation hearing Thursday, and used the occasion to direct comments toward Pakistan. Tensions are high in the wake of the successful U.S. raid on Usama bin Laden's Pakistan compound. Some lawmakers are convinced Pakistani officials must have known bin Laden was hiding there and have questioned the status of the U.S.-Pakistan relationship -- Panetta shared those concerns, but said the two countries must continue working together.
"The relationship with Pakistan is at the same time one of the most critical and yet one of the most complicated and frustrating relationships that we have," he said. "It is critical because ... vital supply lines go through their country. It's critical because they are a nuclear power, and there's the danger that those nukes could wind up in the wrong hands.
"At the same time, it's very complicated, complicated by the fact that they maintain relationships with certain terrorist groups, that they continue to not take aggressive action with regard to these safe havens, and that they're concerned about the sovereignty results and criticisms of the United States when in fact my view is that the terrorists in their country are probably the greatest threat to their sovereignty."
Panetta concluded by saying the U.S. must do "everything we can to try to strengthen that relationship so that both of us can work to defend both of our countries."
Fox News' Justin Fishel contributed to this report.