The way his big brother tells it, Jim Harbaugh never minded the regular moves every few years for his father's football coaching jobs because he needed a change in towns to find fresh surroundings and make new friends given how quickly he ran them off.

Fast forward about four decades and Harbaugh is in the fourth year of his $25 million, five-year contract coaching the San Francisco 49ers, and the relationship might be wearing thin despite success on the field. Harbaugh carries on amid speculation he is grating on everyone from his players to the front office and could be on his way out.

Not that Harbaugh pays much attention to the constant, weekly scuttlebutt. Or, if he does notice, he walks through the hallways and locker room at new Levi's Stadium seemingly unfazed.

"Some people hate you, some people love you, and most of those people don't even know you," he said this past week. "You take it with a grain of salt, smile, and go about your business."

He doesn't have to look far to get a lift. Harbaugh has a long list of well-used quirky phrases, and plenty of other inspirational quotes hanging on the walls at team headquarters. He recites politicians, military men and former coaches like his father and Bo Schembechler.

"I was, and still am, happier than a pig in slop," he revealed a day after turning 50 last December and following a thrilling Monday night win against Atlanta in the NFL finale at Candlestick Park.

To start a new season, he will offer, "They're cheerful and undefeated" to describe the rookies.

So far this season, it hasn't always been so cheerful. Harbaugh has dealt with serious issues, from domestic violence allegations against defensive lineman Ray McDonald and linebacker Aldon Smith's nine-game suspension. Still, the 49ers were 4-3 going into their bye weekend and very much in the chase for the NFC West crown — again.

When analysts Deion Sanders and Trent Dilfer recently weighed in on the 49ers coach, questioning whether the players were still behind Harbaugh, he didn't engage in a back and forth but made his point.

"I haven't seen Trent or Deion around much," Harbaugh said.

What makes Harbaugh tick? That's simple: winning.

Two Novembers ago, he promoted Colin Kaepernick to starting quarterback when Alex Smith had been on one of the best rolls of his career before suffering a concussion.

As far as the critics, Harbaugh might just shrug and shout, "Attack this day with an enthusiasm unknown to mankind!" or "Who's got it better than us? No-body!"

"From what I remember, it was never easy," Harbaugh's sister, Joani Crean, said by email. "He fought and competed for every spot he ever earned. I will say this, his sheer love for the game and competition always got him through anything thrown his way. You have to love what you do to not be shaken. John and Jim both epitomize this for sure."

Keeping things light helps, too.

On Monday, when a pair of Oakley sunglasses were left near the podium where Harbaugh had just finished his news conference, Harbaugh pulled the shades on and quipped, "These glasses are 1985, '90, I used to wear these," then set them back down, left and returned to work.

He even provides details on how toddler son Jack — named after Harbaugh's father — is making big strides physically (like a future athlete, of course).

Little Jack, 2, just "swam across the entire swimming pool, the long way. Very proud," Harbaugh offered in a short conversation with The Associated Press.

"I say, 'Jack, are you going to be a football player?' He says, 'Yes, daddy,'" Harbaugh said.

Not everybody agrees with Harbaugh quite so well.

Whether he and CEO Jed York will find common ground is unclear. York said after last year he hoped to sit down with the coach this past offseason to work out a new deal.

It's difficult to imagine Harbaugh would be back in 2015 without one, serving as a lame-duck coach in the final year of his contract. If he doesn't return, Harbaugh would be in high demand. His alma mater, Michigan, might be in the market for a new coach, as well as the team that gave him his coaching start, the Oakland Raiders.

Some believe he has too much pride to go back to the college game without a Super Bowl ring. Especially because brother, John, and biggest rival, Pete Carroll, both have one — each beating the Niners on the way to a championship.

The 49ers have reached three straight NFC title games.

Harbaugh became restless at Stanford even while pulling off a remarkable turnaround of the Cardinal program, looking into jobs at Kansas and Notre Dame before finally leaving after his fourth season for the NFL and the Niners.

Whether going back and forth with Seattle's Carroll in college and the NFL, or for that infamous backslap of Jim Schwartz in Detroit during his first season of 2011, Harbaugh is so wrapped up in his own world that he often infuriates opponents — and sometimes he might not even realize it.

It has been happening this way since his boyhood years.

"He would alienate the other kids, so I was really the only friend he had," John Harbaugh recalled a few years back. "We joke that dad's profession was the perfect profession for Jim, because after two years, he'd be like: 'It's time to move, dad. I've lost all my friends.' We were in Iowa one time and dad felt bad because we were leaving for Michigan. He tried to break it to us, and Jim goes: 'Just in time, dad. I just ran out of my last friend.'"

Not that Jim's counting who is on his side.

"Again, you take things with a grain of salt," he said, "especially from people that don't know you or have some other agenda that they may be serving. I take it with a grain of salt and go about doing our jobs."

___

AP NFL website: www.pro32.ap.org and www.twitter.com/AP_NFL