No matter that Brooklyn-born Ginsburg and O'Connor, raised on a ranch in Arizona, look and sound nothing alike.
The confusion arose because, even at the dawn of the 21st century, women on the court were "one- or two-at-a-time curiosities," Ginsburg said.
So she considered it progress that no one made that error after Sonia Sotomayor became a Supreme Court justice last year.
Now with Elena Kagan set to join them on the bench for the start of the high court term in October, Ginsburg perceives an even bigger change.
"We are one-third of this court," Ginsburg said during an interview with The Associated Press in her Supreme Court office. No longer a momentous event, the appointment of a woman to the high court has become, Ginsburg said, "expectable."
"I don't think anybody's going to confuse Justice Kagan, Justice Ginsburg or Justice Sotomayor," she said.
But having three women on the court may not change the outcome of any cases. The justices, after all, regularly divide 5-4 along ideological lines in high-profile cases. Sotomayor's votes in her first year were very similar to Justice David Souter's, the man she replaced. Kagan is expected to vote much like Justice John Paul Stevens, who retired in June.
"Having this seat occupied by a woman does not in and of itself change the way this justice votes," said Vanderbilt University law professor Tracey George.
Academic studies have so far found just one area, sex discrimination lawsuits, in which the presence of a woman on a panel of federal appeals court judges appears to make a difference. A three-judge panel that includes a woman "is significantly more likely to rule in favor of" a person claiming sex discrimination, Christina Boyd, Lee Epstein and Andrew Martin concluded in a 2008 paper.
Adding another woman might not change the outcome of cases, but it could have an effect on how the court goes about its business, George said. She cited social science research that suggests the presence of a woman in a decision-making group influences the behavior of others in the group.
Ginsburg put a similar thought plainly. "We do bring to the table the experience of growing up as girls and women," she said.
The 77-year-old justice picked out one case that the court decided in 2009 to illustrate her point. A 13-year-old girl complained about being strip-searched by officials at her middle school in Arizona in pursuit of prescription-strength ibuprofen.
"The initial reaction of the men was, 'What's so terrible? Boys disrobe,"' she said. "But I think the court really appreciated that there is a difference between the reaction of a 13-year-old girl and 13-year-old boy to that kind of exposure."
Ginsburg didn't explicitly say so, but she appeared to be taking credit for changing some minds. The justices voted 8-1 that the search violated the student's constitutional rights.
She also suggested that women were more likely to add a measure of civility to the court's work. Opinions by the court's women "have no nasty comments whether they're writing for the court or in dissent."
Not so for some of the men. She said Stevens was fond of calling others' opinions "profoundly misguided." Justice Antonin Scalia, her good friend on the court, is known for his acerbic writings, which Ginsburg conceded might be more attention-grabbing than others'.
Ginsburg is fond of her service with O'Connor, who retired in 2006. In disagreeing on some major issues, they showed that women "come in all sizes and shapes just like men do."
Kagan and Sotomayor were both nominated by President Barack Obama, a Democrat.
"I'd feel better if there was a conservative woman on the court as well, just so there wasn't an opportunity to think they're liberal just because they're women," said Catholic University law professor Amanda Leiter, a one-time clerk for Stevens.
O'Connor played a critical role on the court, never predictably conservative or liberal, often providing the decisive vote in closely divided cases.
The three women justices "may be in dissent more often" because they make up three-fourths of the liberal side of a more conservative court, Leiter said.
Their Democratic ties, though, could make it more likely that the next Republican president will look to nominate a woman, said Notre Dame law professor and former Supreme Court law clerk Richard Garnett.
Ginsburg said Garnett could be right. "I don't think that when there's the next vacancy the president will feel any compulsion to appoint another woman, but also won't resist appointing another woman," she said.