"It's right and wrong, up and down," said Doherty, 27, when asked by a British Broadcasting Corp. interviewer about the status of the relationship.
But he added: "She's had enough, I think.
"I love her bones, I always will."
In an interview with BBC talk show host Jonathan Ross, to be aired Friday, Doherty also said Moss pressured him to kick his heroin habit permanently.
"She said, 'Give up the smack, and never go on Jonathan Ross again!'" the Babyshambles frontman joked.
Excerpts from the interview were released Wednesday by the BBC.
Moss and Doherty's relationship hit the headlines in September, after pictures of the model allegedly using cocaine appeared in British tabloid The Daily Mirror. Moss, 32, was snapped in the west London music studio where Doherty was recording with his band.
The singer-songwriter also discussed his drug addiction in the interview.
"For the first time in my life, I'm upping the stakes in my battle against (heroin)," he said. "Being clean means I can sit down and rediscover writing. My songwriting suffered, being on drugs."
Doherty recently signed a deal with Orion Books to publish poetry, memoirs and drawings culled from 20 volumes of his diaries.
The rocker is undergoing drug treatment at the order of a British court, following a December arrest in which police discovered marijuana, heroin and crack cocaine in his car and clothing.
Swedish police fined Doherty at the end of May, after traces of cocaine were found in his blood at a music festival.
"I get tested twice a week for crack and heroin, that's part of my court order," Doherty said in the interview, adding that he was trying to stop. "Being skint (broke), drunk, paranoid — no, I don't wish that for myself."