By Paul Virgo
Williams, playing her first tournament since winning January's Australian Open after a knee injury, squandered a match point while leading 5-4.
The American also blew a mini-break advantage in the third-set tiebreaker.
The victory handed twice champion Jankovic her second win over a Williams sister in as many days after she dropped just one game against fourth seeded Venus Thursday.
"I fought very hard," Jankovic told a news conference. "I just tried to stay out there as much as I could.
"Beating two Williams sisters in two days is amazing. I'm really proud of myself."
The match got off to a scrappy start, with the players trading breaks twice amid a raft of double faults and unforced errors.
When things settled down, Williams unleashed three cracking winners to put Jankovic under pressure on serve in the eighth game.
The seventh seed held out on that occasion but Williams grabbed the break she needed in the last game of the set after her opponent mis-hit a shot in the decisive point.
The Serb came back strongly, giving the American no peace on her serve and breaking twice to claim the second set.
Jankovic played catch-up throughout the final set but she refused to lie down.
"I wasn't playing the points right," Williams said. "I feel I could have won. I should have won. I was making errors on the key points.
"I feel I might show up tomorrow and play. I don't feel I lost. I don't think it's set in."
(Editing by Pritha Sarkar)