AUSTIN, Texas (AP) -- The awkward dismissal of Texas football coach Mack Brown came first. That was followed by the firings of school President Bill Powers and basketball coach Rick Barnes.

Now athletic director Steve Patterson is gone, too, after less than two years -- an abrupt end to his rocky tenure atop the nation's wealthiest athletic program that remains in turmoil.

Patterson, whose aggressive approach to raising money rankled fans and some major donors, resigned Tuesday in what university President Greg Fenves called a "mutual" agreement.

More from FoxSports

Fenves refused to say he fired Patterson but said "this has been a decision that's been building over recent weeks," and noted he'd heard from "hundreds if not thousands" of Texas fans concerned about the program.

Patterson's departure came three days after a plane carrying a "Patterson Must Go" banner circled the football stadium before the Longhorns' first home game of the season. His tenure lasted only 22 monthts.

Patterson was responsible for two popular hires -- football coach Charlie Strong and basketball coach Shaka Smart -- but also raised ticket prices after a losing football season, pushed coaches to raise money for endowed scholarships and created a fan "loyalty" program to prioritize ticket purchases.

And while Strong still enjoys the support of fans, he is 7-8 overall in his second season. Smart hasn't yet coached a Texas game.

"Change is never easy, but I have every confidence that UT Athletics will continue to thrive as it embraces the future" Patterson said in a statement released by the school.

Firing Patterson creates a major void at a Big 12 power at a time when some league presidents are pushing to expand from the current 10 members, and trying to solidify their position in the new College Football Playoff. And it comes as Texas is seeking a lucrative new apparel contract once its current deal with Nike expires in 2016. Nike recently partnered with Michigan in a deal worth a reported $169 million.

Fenves previously met several times with Patterson after complaints from fans and donors, which had reached as high as Board of Regents Chairman Paul Foster.

Patterson has a guaranteed contract through August 2019 that pays at least $1.4 million per year. Fenves said a negotiated separation deal will go to the board of regents for approval.

Fenves said Texas will hire former Texas football player Mike Perrin, a Houston attorney, as interim athletic director on a $750,000 contract through August 2016.

Patterson was supposed to be a hire that would calm turbulent waters at Texas.

The university had put together a star-studded committee to find a replacement for DeLoss Dodds, a powerful figure who retired in October 2013. An impressive interview won Patterson the job over former West Virginia athletic director Oliver Luck, who many had considered the favorite for the job.

Pressure to fire Brown was already building to fever pitch before Patterson arrived in November 2013. Brown was fired barely six weeks later in the first of several major departures.

Powers, a key player in the Big 12, was forced to resign in 2014 after years of clashes with regents. By March 2015, Patterson had fired Barnes even after the coach led the Longhorns to the NCAA tournament for the 16th time in 17 seasons.

Patterson broke barriers by hiring Strong away from Louisville, making him the first black head football coach at Texas, which had a long history of segregation and racism. He also pushed to get alcohol sales at athletic events, scheduled a men's basketball game in China next season and talked of someday scheduling an international game for the Longhorns football team.

Most of Patterson's career was spent as an executive in professional sports, notably the NBA, and he struggled to connect with university officials and supporters in the same way as Dodds.

One of his first missteps was an awkward public push to have the city of Austin help finance a new basketball arena after having not "invested a nickel" in the current Erwin Center over the previous 30 years. Those comments caught city officials off guard and forced the school to backtrack.

And several actions raised questions of whether he connected with his coaches.

At his exit news conference, Barnes accused Patterson of leaking private conversations and demands that Barnes shake up his staff before he was ultimately fired.

Patterson also refused to engage in a breach-of-contract lawsuit filed by Oklahoma State against Texas offensive line coach Joe Wickline. Texas was not named as a party in the case, which is still on-going, but led to Strong and his staff being subjected to embarrassing depositions to describe who calls plays.