Mularkey, who went 14-18 as coach of the Buffalo Bills from 2004-05, is Khan's first major hiring as new owner.
"The Jaguars have hired Mike Mularkey as head coach. [A] press conference will be held on Wednesday," the team announced on its Twitter page late Tuesday.
The 50-year-old has spent the last four seasons with the Falcons, overseeing the development of quarterback Matt Ryan since the Falcons drafted Ryan third overall in 2008, FOX Sports reported.
Mularkey actually interviewed for the position Tuesday, according to the report, after Atlanta was eliminated from the playoffs by the New York Giants on Sunday in a 24-2 loss in the wild-card round.
His first priorities will be assembling a staff and meeting with quarterback Blaine Gabbert, who just finished his rookie season.
Mularkey becomes the fourth head coach in Jacksonville's 17-year history. He replaces interim coach Mel Tucker, who went 2-3 over the final five games of the season after Jack Del Rio was fired by former majority owner Wayne Weaver in late November.
Del Rio was 68-71 in eight-plus seasons with the Jaguars, taking them to the postseason twice and going 1-2 in the playoffs.
The Times-Union reported that Mularkey was also scheduled to interview for the Miami Dolphins' head coach opening this week. Mularkey coached in Miami the two seasons before going to Atlanta.