The White House on Thursday urged Indiana to back off a new law that would withhold taxpayer funding to Planned Parenthood, setting up a clash with the state after formally declaring its policy illegal.

White House Press Secretary Jay Carney and Medicaid administrator Donald M. Berwick both defended the administration's decision to reject the Indiana law. They both said they expect Indiana to comply, declining to say whether the administration would withhold federal money should Indiana defy the order.

"We expect Indiana will comply with the federal law. That's our position now," Berwick told Fox News. "Medicaid can't pay with federal dollars for abortion, but that does not mean the state can deny services from a willing provider. That's just what the law is and we're just implementing the law."

The Indiana Legislature voted in April to cut off $3 million in federal money the state distributes to the family planning group. Supporters of the bill argued that if the state gives money to Planned Parenthood to use on services other than abortion, then Planned Parenthood has that much more of its own money to use on providing abortions. Federal law prohibits the use of taxpayer money for abortion services.

In the letter, Berwick gave Indiana 60 days to challenge the decision. The letter does not state it explicitly, but Indiana could be at risk of losing $4 million in federal Medicaid money if it does not comply. In the past, state Medicaid plans that did not conform with federal law have been changed by states before the U.S. Health and Human Services Department enforced any penalties.

Pointing out this history of compliance, Carney said the administration would "work with Indiana" on the dispute. "(We) fully expect Indiana to come into compliance," Carney said.

But Bryan Corbin, a spokesman for Indiana Attorney General Greg Zoeller, said the state plans to stand by the law.

"We are reviewing the CMS letter with our client (the state health office) ... to determine our client's options, but we will continue to defend the statute," he said in an email to FoxNews.com.

Indiana Republican Rep. Marlin Stutzman claimed the state was "well within both the letter and spirit of federal guidelines" and predicted Indiana officials would appeal.

"I take it very seriously when any administration tells a state that it cannot administer its own programs. Health and Human Services should respect the will of Hoosiers who simply ask that their tax dollars do not subsidize an entity that maintains an abortion clinic," Stutzman said.

The back-and-forth comes after Berwick said in a letter to the state's Medicaid director that Indiana's plan would improperly bar beneficiaries from receiving services.

Federal law requires Medicaid beneficiaries to be able to obtain services from any qualified provider.

"Medicaid programs may not exclude qualified health care providers from providing services that are funded under the program because of a provider's scope of practice," Berwick wrote to Patricia Cassanova, the director of Indiana's office of Medicaid Policy and Planning. "Such a restriction would have a particular effect on beneficiaries' ability to access family planning providers."

In addition to Berwick's letter, HHS also posted a notice on Wednesday that sought to make clear that the department would take a dim view of similar efforts in other states to ban specific providers from federal funds. At least four other states have taken a similar tack as Indiana's, arguing that they are not preventing Medicaid recipients access to services by prohibiting Planned Parenthood from receiving federal dollars.

The HHS notice, written by Cindy Mann, the director of the Center for Medicaid, CHIP and Survey & Certification, emphasizes that states may bar providers from participating in Medicaid in certain circumstances, such as if a provider is committing fraud or criminal acts.

"States are not, however, permitted to exclude providers from the program solely on the basis of the range of medical services they provide," Mann wrote.

Medicaid is a federal-state partnership that nationwide now covers more than 60 million low-income children and parents, seniors, including most nursing home residents, and disabled people of any age.

While Planned Parenthood provides abortion services, it also provides other services such as preventive care and family planning and is eligible to receive Medicaid funding for its other services. Planned Parenthood operates 28 clinics in Indiana, four of which perform abortions, the organization says. The state chapter has said federal funding makes up about 20 percent of its annual budget.

Fox News' Molly Henneberg and The Associated Press contributed to this report.