Aijalon Gomes, 31, hugged Carter just before they boarded a plane at Pyongyang's airport, footage aired by broadcaster APTN in North Korea showed.
Carter had flown to the North Korean capital three days earlier on a rare private mission to negotiate Gomes' release.
The former president "courteously requested" a special pardon for Gomes, which leader Kim Jong Il granted, North Korean state media said. Gomes had been sentenced in April to eight years of hard labor and a hefty fine for trespassing and committing a "hostile act."
They were due to arrive in Boston on Friday to be reunited with Gomes' mother and other family members, Carter spokeswoman Deanna Congileo said in Atlanta.
In Washington, the State Department welcomed the news of Gomes' release.
We "are relieved that he will soon be safely reunited with his family," State Department spokesman P.J. Crowley said. "We appreciate former President Carter's humanitarian effort and welcome North Korea's decision to grant Mr. Gomes special amnesty and allow him to return to the United States."
North Korea's state-run Korean Central News Agency said Carter's visit included cordial talks with North Korea's No. 2 official, Kim Yong Nam.
Kim relayed Pyongyang's interest in resuming the six-nation disarmament talks and reiterated the regime's commitment to denuclearization, KCNA said.
However, there was no indication that Carter met with Kim Jong Il, who was making a surprise trip to China during the rare visit by an American dignitary to the North Korean capital. A year ago, Kim sat down for talks and a well-publicized photo with former U.S. President Bill Clinton when he went to Pyongyang on a similar journey to negotiate the release of two American journalists.
The U.S. and North Korea fought on opposite sides of the 1950-53 Korean War and do not have diplomatic relations, but Carter is well-regarded in North Korea after making a groundbreaking trip to Pyongyang in 1994 to meet with Kim's father, late President Kim Il Sung. Those friendly talks led to a landmark nuclear disarmament pact.
Aijalon Gomes (pronounced EYE-jah-lahn GOHMS) was the fourth American arrested in North Korea for illegal entry in a year when he was seized in January, accused of crossing into North Korea from China.
Besides journalists Laura Ling and Euna Lee, also sentenced to hard labor and released last August on a special pardon, activist Robert Park deliberately crossed into the country last Christmas and was expelled by North Korean authorities about 40 days later.
Gomes' motive was not clear but he was said to be a close friend of Park and was photographed in Seoul rallying for Park's release just two weeks before his own arrest in North Korea.
In April, North Korean authorities sentenced Gomes to eight years of hard labor and fined 70 million won -- more than $600,000 -- for trespassing and committing a "hostile act." Gomes "admitted all the facts," state-run media said.
Last month, North Korean media reported that Gomes tried to kill himself, "driven by his strong guilty conscience, disappointment and despair at the U.S. government that has not taken any measure for his freedom," and was hospitalized.
A U.S. delegation, including a consular official, two doctors and a translator, made a secret visit to Pyongyang earlier this month to try to secure Gomes' release. The group visited Gomes at the hospital but were unable to negotiate his release then, State Department spokesman P.J. Crowley said last week.
Tall and muscular when he appeared at anti-North Korean rallies in Seoul in January, Gomes looked markedly thinner Friday. He was dressed in a white striped polo shirt and dark slacks, APTN footage showed.
"The measure taken ... to set free the illegal entrant is a manifestation of (North Korea's) humanitarianism and peace-loving policy," KCNA said.
The Carter Center and U.S. officials have emphasized that the ex-president's trip was a private humanitarian mission.
However, such visits have in the past provided an opportunity for unofficial diplomacy.
KCNA said the Americans held "an open-hearted discussion" with North Korea's foreign minister and vice foreign minister for U.S. affairs on their countries' relations as well as nuclear disarmament.
Six-nation nuclear disarmament talks have been stalled since North Korea walked away from the table last year.