President Obama said it's not him or Democrats who are holding up comprehensive immigration reform, it's Republicans. The president emphasized at remarks to the National Council of La Raza in Washington, D.C. Monday.

"[F]eel free to keep the heat on me and keep the heat on Democrats. But here's the only thing you should know. The Democrats and your President are with you. Don't get confused about that. Remember who it is that we need to move in order to actually change the laws," he said.

Obama told the group there is "unfinished business" and that "I promised you I would work tirelessly to fix our broken immigration system and make the DREAM Act a reality." The DREAM Act, which would give citizenship to children of immigrants who are in college or serving in the military, failed in the Senate last year and has not had success moving forward yet in 2011.

"Believe me...the idea of doing things on my own is very tempting," he said, adding because that's not how things work with the constitution.

"Yes you can, Yes you can!," the crowd chanted in response.

"So let's be honest. I need a dance partner here -- and the floor is empty," he added.

The president noted, what he sees as problems in the system - that America allows "best and the brightest" to study, but then asks them to leave, that it "tolerates" immigrants and business that break rules, and punishes those that follow.

"I swore an oath to uphold the laws on the books, but that doesn't mean I don't know very well the real pain and heartbreak that deportations cause. I share your concerns and I understand them. And I promise you, we are responding to your concerns and working every day to make sure we are enforcing flawed laws in the most humane and best possible way," Obama said.

After the speech, the Director of the National Day Laborer Organizing Networked fired off a statement calling the president "deporter-in-chief" and saying they feel he's starting to go "rogue."

"Despite soaring rhetoric, the president's unbridled enforcement of unjust and outdated immigration laws has contributed to an unprecedented civil rights crisis for our community. And his administration has deported over one million people, surpassing the total number of people removed during Operation Wetback. The President can now claim the title, deporter-in-chief," said Director Pablo Alvarado.

Obama vowed to keep up the fight for legislation, but didn't give a timetable for the next action. Earlier this year in El Paso, Texas he presented his vision for immigration reform.