Mayor Bill de Blasio said the closure will last through at least April 20 and possibly for the whole school year.
The decision came amid a growing number of school closures in communities and entire states across the U.S. as well as mounting pressure from New York residents, City Council members and others.
The mayor called it a "very troubling moment, a moment when I'm just distraught at having to take this action, but I became convinced over the course of today that there is no other choice."
He also announced that the number of deaths in the city rose to five and he was ordering the end of elective surgeries.
New York City has the largest public school system in the nation, with nearly 1,900 public schools. Many private schools already have closed.
Gov. Andrew Cuomo's office had announced the city school closure moments before de Blasio, but initially said it would start as early as Tuesday.
De Blasio initially was reluctant to close the school system because of the consequences for students and their families. Just this past Saturday, the mayor, a Democrat, said it was critical to keep schools open.
He said he worried that health care workers and first responders would have to stay home to care for children, and that hundreds of thousands of poor students could go hungry without their free or reduced-price school meals.
Multiple states already have announced they were closing schools, as have major cities including Los Angeles, Houston, San Francisco, Seattle and Washington, D.C.
The shutdown had started to seem inevitable Sunday as de Blasio lost key support to keep schools open and Cuomo called for all downstate schools to be closed.
The Associated Press contributed to this report.