San Diego Chargers quarterback Philip Rivers (center) celebrates with teammates after the Green Bay Packers were given a penalty on fourth down during the second half Sunday, Oct. 18, 2015, in Green Bay, Wis.

Mike Roemer AP

SAN DIEGO (AP) -- Philip Rivers threw for 503 yards and the San Diego Chargers still couldn't pull off an upset of the Green Bay Packers at Lambeau Field.

Now the Chargers (2-4) turn their attention to the AFC West starting with Sunday's home game against the Oakland Raiders, who are at once a longtime rival and a possible partner in a proposed stadium in an industrial suburb of Los Angeles.

More from FoxSports

"The quickest way to the playoffs is winning your division," safety Eric Weddle said Monday. "It starts now. Hopefully we can get on the right step."

One big problem, though, is that four-time defending division champion Denver has jumped to a 6-0 start, including 2-0 in the AFC West.

San Diego finished 2-4 in the division last year, winning only the two games against last-place Oakland.

Sunday's game could be the last time the Raiders visit San Diego.

While fans of the teams still hate each other, Chargers chairman Dean Spanos and Raiders owner Mark Davis have proposed to jointly build a stadium in Carson if they can't get new stadiums in their home markets. The Chargers and Raiders were spooked by St. Louis Rams owner Stan Kroenke announcing intentions to build a stadium in Inglewood.

Chargers officials walked away from negotiations with San Diego officials in mid-June.

Oakland is 2-3.

The Chargers will have to shake off a gut-wrenching 27-20 loss to the Packers that was sealed when rookie cornerback Damarious Randall poked away Rivers' pass to Danny Woodhead near the front right pylon in the final seconds.

Rivers set franchise marks with career highs of 43 completions, 65 attempts and 503 yards passing. He threw for two touchdowns. His 43 completions were the most in NFL history in a loss.

"It's unfortunate he had such an amazing day and we didn't come up with the win," said Weddle, who injured his groin late in the game. "We've all got to play a little better for him, to help him, make things easier on him. He, like everyone else, has to get better, myself, everyone, we all need to get a little bit better.

"We'll start winning these games and get on a run here."

Chargers rookie running back Melvin Gordon had a miserable homecoming to his native state. He fumbled twice, losing one, and was benched.

Gordon has fumbled four times this season, losing three.

"We've got to eliminate it," coach Mike McCoy said. "We told him today, we told him yesterday during the game, what he's got to do. He can't put the ball on the ground."

Will Gordon start Sunday?

"We'll do what's best for the football team moving forward," McCoy said.

Gordon has yet to score his first NFL touchdown.