By Simon Cambers
The fourth-seeded Scot served for a two sets to love lead but then suffered a left thigh problem and Wawrinka, who also had treatment for a thigh injury, recovered to win after three hours 56 minutes.
The defeat handed Murray his earliest defeat at a grand slam event since he was beaten at the same stage in the French Open in 2008.
Murray played down the injury but admitted he had begun to feel some tightness in his left thigh early in the second set.
"In the third and fourth sets, I was struggling physically and I got frustrated with that," Murray told reporters.
"I haven't been in that position for a very long time (having fitness issues). Maybe I felt like my chance of doing well here was slipping away.
"I just didn't feel great. There were a lot of things that I was feeling on the court. I just haven't felt that way for a few years now."
Murray, the runner-up in New York in 2008 and also at the Australian Open this year, began slowly but bounced back from 5-2 down to win the first set on a tie-break.
The Scot stormed to a 3-0 lead in the second set and though he was broken in the fifth game, he then broke Wawrinka again to lead 5-3.
At 5-4 and 30-30 on his own serve, he was two points from taking a two-set lead but Wawrinka became more aggressive and after breaking back, ultimately snatched the tie-break 7-4 to level the match.
Murray then broke in the opening game of the third set but immediately dropped his serve and in the next game, it seemed that he was struggling with his movement, especially to his left.
Murray's service speed also dropped and Wawrinka then broke again in the fourth game on his way to a 4-1 lead, before Murray had treatment on his left thigh.
The Swiss then pulled up in the next game and had to have his right thigh strapped but still it was Murray who was struggling the most and Wawrinka held on to take the set.
The pair traded breaks at the start of the fourth set but Wawrinka broke again in the fifth game and though Murray saved one match point, a forehand long handed the Swiss another break and the match.
"I just tried to focus on being really aggressive," an elated Wawrinka said.
Wawrinka will play American 20th seed Sam Querrey in the fourth round.
(Editing by Steve Ginsburg)