Millsap had a dunk and a fallaway jumper to help the Jazz finish the game on a 10-4 run.
Russell Westbrook had 23 points for Oklahoma City, and Kevin Durant had 18, but went scoreless in the fourth quarter and misfired on all seven attempts from the field in the period.
James Harden added 17 points for the Thunder, who have lost four of their last seven games. The NBA's most efficient offense went 6 of 21 in the fourth quarter and committed 20 turnovers in the game.
After Derrick Favors made two free throws for an 85-77 Utah edge, Westbrook made two 3-pointers and a pair of foul shots to power a 9-2 run and cut Utah's lead to 87-86 with 2:56 to play.
Jamaal Tinsley scored nine points in 9 minutes and Utah's reserves turned a deficit into an eight-point lead during the second quarter. The Jazz led 52-44 at the half on the strength of 11 forced turnovers.
Utah extended the advantage to 14 in the third quarter. Harris scored 13 straight points for the Jazz and opened the middle for Millsap and Jefferson to operate.
Oklahoma City's Serge Ibaka and Harden hit several wide-open jumpers as Utah's big men hurried to help stop the drives of Durant and Westbrook to get the Thunder back within striking distance.
The Jazz have won six straight games at home and looked nothing like the team that lost two games within a week in mid-February to Oklahoma City by an average of 20 points.
After a string of five consecutive losses to Oklahoma City and looking outmanned by the Thunder's length and quickness, Utah employed a scrambling, helping defense that forced difficult shots and bad passes.
The Jazz outscored the Thunder 50-20 in the paint and moved a step closer to reaching the playoffs. Utah is ninth in the Western Conference, a half-game behind Houston for the last spot.
After an MRI earlier Tuesday, the Jazz learned that starting forward Josh Howard's knee might require surgery and he could be lost for the season. Howard said he might seek a second opinion.
Notes: Utah guards Raja Bell (strained left adductor) and Earl Watson (sprained left ankle) sat out. ... Oklahoma City's Daequan Cook limped off with a right knee sprain early in the fourth quarter and did not return. ... Tinsley dribbled the ball through Mohammed's legs and tossed in a floater in the fourth quarter.