The donation reportedly came with a comment that read: "God bless. Thank you for your courage. Keep your head up. You’ve done nothing wrong."
"Every rank and file police officer supports you," the message continued. "Don’t be discouraged by actions of the political class of law enforcement leadership."
Norfolk City Manager Chip Filer on Tuesday said he’d reviewed an internal investigation of the incident and determined that Kelly violated city and departmental policies, local outlet WAVY-TV reported.
"His egregious comments erode the trust between the Norfolk Police Department and those they are sworn to serve," Filer said in a statement Tuesday.
"The City of Norfolk has a standard of behavior for all employees, and we will hold staff accountable."
In his own statement, Police Chief Larry Boone said that the force doesn’t "want perceptions of any individual officer to undermine the relations between the Norfolk Police Department and the community."
Kelly was an 18-year veteran of the department and had been the executive officer of internal affairs for the last two months, the report said.
He can appeal the decision about his termination.
Rittenhouse, 18, has been charged with murder and attempted murder for shooting and killing two people and wounding a third at a protest in Kenosha, Wisconsin on Aug. 25.