"It's been the most difficult time of my life, no two ways about it, because every single person believed that I should know or did know or had something to do with it," Williams said. "I knew nothing, that's my answer. I don't have to clarify or extend that answer, I knew nothing."
The New Zealander, who has been Woods' caddie for 11 years, said he would have spoken up if he had known about the player's behavior.
"If the shoe was on someone else, I would say the same thing. It would be very difficult as a caddie not to know but I'm 100 percent telling you, I did not know, and that's that," he said. "I'm a straight-up sort of person. If I had known something was going on, the whistle would have been blown."
Williams said he was angry when revelations about Woods' private life emerged, but had not berated the player because he felt he needed a friend.
"Of course I'm mad at him, why would you not be?" Williams said. "I'm close with his wife and he's got two lovely children and he's let them down.
"When a guy's having a tough time, it's not up to me to beat him with a stick right now. He's getting enough grilling from everybody else.
"When you're a true friend of somebody, that's when somebody needs your support and need you the most. That's when you don't walk away. Tiger's one of my closest friends and he needs my support right now and I'd never think of walking away."
Williams said the two haven't discussed the scandal.
"When I talk to him, I don't talk to him about what's happened," Williams said. "I talk to him about the future and about what we're going to try to accomplish and how we're going to get over it."
Williams said Woods recently hit balls on the practice range, but would not return to golf until he felt he was in top form.
Williams said he had personally tried to concentrate on his charity work and auto racing interests to avoid thinking about the controversy around Woods.
"Every week I try to focus on something to keep my mind off it," Williams said. "You try to deal with it as best you can but in some peoples' perception, I'm involved in it, I've committed a crime, I've done wrong or whatever it may be."