The Duke coach was responding on a radio show Tuesday to a question about remarks made by Rose during a recently aired ESPN documentary about Michigan's Fab Five teams, which went to two Final Fours in the early 1990s.
Initially, Krzyzewski said he didn't want to comment when asked about it on ESPN 1000 in Chicago, but he followed with his criticism.
"I just think that sometimes people say things to get attention so that they can sell their documentary," Krzyzewski said. "Obviously, that was a poor choice of words and very insulting to everyone here at Duke but especially, not just our African-American players, but any African-American students."
Krzyzewski called the Fab Five "good kids," but he suggested they had nothing to show for it.
"They had a heck of a run, but they didn't leave anything," he said. "They didn't establish anything there."
The Fab Five lost the 1992 national title game to Duke, then fell to North Carolina in the 1993 championship game. Eventually, Michigan forfeited its wins after it was revealed that some players, including star Chris Webber, had accepted money from a Michigan booster.
"The guys that I had established something that Jay Williams continued to do 10 years later," Krzyzewski said. Williams, a point guard for Duke from 1999-2002, was part of the radio interview.
"I hated everything I felt Duke stood for," Rose said in the documentary, describing his feelings as a 17-year-old high schooler. "Schools like Duke didn't recruit players like me. I felt like they only recruited black players that were Uncle Toms."
Hill criticized Rose's comment in a column that appeared on The New York Times website.