Carter's three-day visit, which featured two days of talk with President Raúl Castro, is intended to boost strained relations between the United States and its Caribbean neighbor.
Members of Cuba's small dissident community, including internationally renowned blogger Yoani Sánchez and Oswaldo Paya, who sought a referendum on the communist government and guarantees of such rights as free speech and private business ownership, confirmed that they, among others, were invited to the breakfast meeting in historic Old Havana.
Carter was scheduled to give a news conference Wednesday afternoon before leaving the island.
Cuban opposition members have achieved certain fame overseas for their marches, hunger strikes and other activities to push for political and economic change. Visiting U.S. officials often meet with them, although it rankles the Cuban government.
They are not as well known on the island, however, and the government calls them mercenaries paid by Washington to undermine the government. Recently a U.S. diplomatic cable revealed by Wikileaks described the dissidents as riven by petty rivalries and hopelessly out of touch, with leaders too focused on funding their operations to mount a serious opposition.
Meanwhile, there had been speculation that Carter would leave Cuba with imprisoned U.S. contractor Alan Gross, who was recently sentenced to 15 years in prison after being convicted of illegally importing communications equipment. There has been no sign of a resolution to that dispute between Washington and Havana, however.
Gross was arrested in December 2009 while working on a USAID-backed democracy-building project and convicted of crimes against state security earlier this month in a case that has blocked improved ties between the U.S. and Cuba. He has said he was working to improve Internet communications for Cuba's tiny Jewish community, though community leaders deny knowing or working with him.
Havana considers such USAID projects to be aimed at toppling the government.
Carter's visit came soon after the sentencing, and its sudden announcement sparked expectations that he might help facilitate Gross' release. Last August, the 39th U.S. president and winner of the 2002 Nobel Peace Prize traveled to North Korea to secure the release of an imprisoned American.
He said Tuesday that he discussed the Gross case with Cuban officials but was visiting to talk about strained ties.
"I am not here to take (Gross) out of the country," Carter said in Spanish.
"We are here to visit the Cubans, the heads of government and private citizens. It is a great pleasure for us to return to Havana," he added. "I hope we can contribute to better relations between the two countries."
Washington and Havana have not had formal diplomatic relations since the 1960s, and the United States maintains economic and financial sanctions on the island. U.S. officials say no thaw in relations is possible while Gross is in prison.
Carter — who was president from 1977 to 1981 — previously visited Cuba in 2002, becoming the only former U.S. president to do so since the 1959 revolution that brought Fidel Castro to power.
Based on reporting by The Associated Press. AP writer Andrea Rodríguez in Havana contributed to this report.