Alex Ovechkin raced in alone on New York Rangers goalie Henrik Lundqvist during overtime with a chance to win it right then and there.

If Ovechkin, one of the NHL's purest goal scorers, had put the puck in the back of the net, it would have been a fitting ending for the Washington Capitals. Instead, he was denied by Lundqvist.

The Capitals got the win anyway — with a fluke goal that finished off a stirring and most unlikely comeback and has put Washington on the verge of the elusive second round.

Jason Chimera scored 12:36 into the second overtime for the Capitals, who wiped out a three-goal hole and beat the Rangers 4-3 on Wednesday night.

Washington was on the verge of two full days of questions about the franchise's poor postseason history that has featured many big collapses. The Rangers stormed out to a 3-0 lead in the second period and had been perfect in 29 chances this season when leading after 40 minutes.

But now the Capitals lead the series 3-1, when 2-2 appeared to be inevitable, and they can knock out the Rangers in Game 5 at home on Saturday.

"It was an exciting game," said Ovechkin, who had his shot at glory turned aside by Lundqvist with 8:59 left in the first overtime. "It changes our mentality right away. We talked about the second period as a group, and it worked. If we play the same way, the way we played in the third period and overtime, no one can stop us."

Washington won the opening two games of the series at home before losing 3-2 in Game 3 at New York on Sunday. An even series seemed to be a lock until the Rangers' third-period meltdown.

Now the Capitals have the unexpected chance to close it out in Game 5. Top-seeded Washington was in this position last year — holding a 3-1 lead over eighth-seeded Montreal — but lost the final three games. Much of the Capitals' poor postseason history has been a topic of discussion that Capitals coach Bruce Boudreau has tried to quell.

In five previous best-of-seven series in which they held a 2-0 lead, the Capitals have lost four.

"Losing an overtime game in the playoffs hurts all the time, but our team will bounce back," Rangers coach John Tortorella said. "We can't look at the hill, we have to win the next game. We have to find a way to win one hockey game and we will see what happens from there."

The Rangers have never come back to win after trailing a series 3-1, but the Capitals have blown four such leads.

Boudreau incurred the wrath of insulted Rangers fans, who fired back after Washington's bench boss said Madison Square Garden wasn't any louder than Washington's home rink, that its locker rooms and benches were "horrible," and that the reputation of the famed arena was much better than the actual building.

Fans yelled disparaging remarks at Boudreau, chanted "BOOOOD-REAU!" and then yelled in unison, "Can you hear us?" when the Rangers took over in the second period.

He surely could, but the silence at the end when Chimera won it stood out even more.

"I might have made a mistake by saying what I said," Boudreau said. "Let's leave it at that. Let's let the players decide this and not worry about how I felt or what the crowd felt like."

Chimera took a shot from near the right circle that was blocked in front. Rangers forward Marian Gaborik raced in and tried to clear the puck out of the crease but swept it right to Chimera.

"It hit my chest and went down," Chimera said. "It felt like forever until it went down to my stick. It was a good effort by us in the third period and what a comeback.

"There was no quit. Once we got one, we felt them sit back a bit. We took advantage of it."

Lundqvist seemed poised to freeze the puck before Gaborik poked it away. Lundqvist doubled over and stayed down on his knees as teammates skated slowly toward him. At the other end, the Capitals stormed off the bench and gathered in celebration with winning goalie Michal Neuvirth, who made 36 saves.

"Right now, it is painful," Lundqvist said. "It is just unfortunate to have that in overtime."

Alexander Semin started Washington's rally, and Marcus Johansson scored twice in the third to make it 3-3.

The Rangers built their big lead on a goal by Artem Anisimov and tallies just 7 seconds apart by Gaborik and Brandon Dubinsky. Lundqvist was the hard-luck loser after making 49 saves.

New York was the only NHL team this season to be perfect when leading after two periods (29-0), but none of that matters now.

"We looked nervous," Rangers coach John Tortorella said. "We felt better once we got through (the third period) and got to overtime. We still had chances we didn't make. We got beat by a goal that is a nothing goal.

"It is just a nothing play that turns into something — obviously something big."

Heading into the third period, the desperate Capitals ditched their defensive-minded ways and ignited the high-flying offense they have long been known.

"We told them to never give up," Boudreau said. "You get one and you never know. When we got both goals quickly, we believed we were in it."

Semin started the comeback at 2:47 with a goal that stood up to a brief video replay. Rangers rookie defenseman Ryan McDonagh gave the puck away to Semin in the right circle, and the Capitals forward fired a shot that got past Lundqvist's leg and slid to the goal line. Lundqvist swept it out, but couldn't cover it before Semin put it in again.

Johansson scored his first just 57 seconds later off a pass from the left wing boards from Brooks Laich.

"You can't give that team two free ones," Tortorella said. "We did and it cost us."

NOTES: The Rangers' playoff record for fastest two goals is 6 seconds when Rod Gilbert scored both in the second period at Chicago on April 11, 1968. ... Capitals RW Mike Knuble sat out after being injured in Game 3. He was replaced by Eric Fehr, who made his first appearance in the series. ... New York's Brian Boyle took two penalties for goalie interference.