This is a partial transcript from "The O'Reilly Factor," Dec. 1, 2004, that has been edited for clarity.

Watch "The O'Reilly Factor" weeknights at 8 p.m. and 11 p.m. ET and listen to the "Radio Factor!"

BILL O'REILLY, HOST: Our "the top story" tonight, the president of the NAACP steps down. According to the "Atlanta Journal-Constitution" Kweisi Mfume's contract was not renewed. But he says he wanted out to spend more time with his teenage son. Mr. Mfume joins us now from Baltimore.

All right.

KWEISI MFUME, RETIRING NAACP PRESIDENT: Hi there, how are you?

O'REILLY: I'm ok.

MFUME: Good.

O'REILLY: I have to ask you these questions. We have a secret source inside the NAACP. All right. This is what our secret source said. It says you and Julian Bond (search) did not see eye to eye over his inflammatory anti-Bush rhetoric. Is that true?

MFUME: Well, I've been accused of making inflammatory Bush rhetoric, so I think more than anything else, what you probably are getting is a person's perspective. In fact, Julian and I have never really talked about his comments regarding the president at the convention, no...

O'REILLY: But compared to Bond, your rhetoric is like Dick Cheney (search), I mean, he's so over the top, so anti-bush, you, yeah, you took jabs at him but you're pretty moderate at least when you come on this program.

MFUME: Well, I have got a lot of respect for you. But I think in the heat of a campaign there's a lot said on both sides. Sometimes it may be over the top. Sometimes it's not. For me the important thing is to recognize as Americans, as we do, that once a campaign is over, it's over. And we find a way to build consensus to move on. And that was the real catalyst behind my letter to the president right after the election.

O'REILLY: And I thought it was very well written and the sentiment I think was a good one. But our source says that you warned Julian Bond that if he continued to bash Bush in a way he has that the IRS would come down and indeed investigate the NAACP, which it's doing. Is that true?

MFUME: Not true.

O'REILLY: Not true?

MFUME: Not true. No. Not true. He's the chairman of the board. I serve with the pleasure of the board. So I'm not going to be running around warning him not to say anything.

O'REILLY: Well, advise him. You say maybe that's not the best thing to do.

MFUME: Nah.

O'REILLY: You never did that? Come on.

MFUME: I didn't do that because I didn't know what he was going to say. We don't look at each other's speeches before we...

O'REILLY: After he said it, though. After he said it, didn't you say, "Hey, maybe that's a little bit over the top?"

MFUME: No, I didn't. I did not say that. And that's the real truth.

O'REILLY: I believe you. You're an honest guy. Gay marriage — our source says — Bond was for it... You, ah, didn't think — Most black Americans are against it. Was that an issue between the two of you?

MFUME: That was not. And I don't want to try to defend Julian. I'm sure if he were here would do it but he was simply against the amending of the Constitution which later became interpreted for him being for gay marriage but his belief was — and it was articulated through our Washington bureau that it is a sacred document and you don't tinker with it and when you do, you ought to be darn certain you're doing the right thing. So that's where the opposition was. But I think it was interpreted around the country as saying, ah, he's for gay marriage. He was really for ...

O'REILLY: So what you're telling me is you and Bond never had any disagreements at all, you guys were just like Laurel and Hardy, just buddies to the end?

MFUME: No. We've had the same amount of disagreements that any two people would have that are strong willed, that are passionate about their work in politics. But the interesting thing is I don't think we had any more disagreements than, say, the president and a member of his cabinet on issues. You have perspectives and you look at those perspectives and when they differ, if you're smart, you try to find a way to reach consensus, so that's been the way we've proceeded but it's been painted some other way.

O'REILLY: One more question on this and I want to get on to others. The Atlanta paper, the "Constitution-Journal" a fine paper — carries my column, so we love them. They say that your contract wasn't renewed. Is that true?

MFUME: Well, the contract expired on October 24. I had said to the board, please don't reconstruct it. Let's not enter into negotiations. I think I really want to move on. And, you know, "The New York Times" reported it accurately today, Mr. Bond confirmed that, several board members confirmed it. That's the way the process actually went.

O'REILLY: If you wanted to come back, do you think they would have given you a new contract?

MFUME: Well, they tried to talk me out of leaving, so my assumption is that they would have, yes.

O'REILLY: Now, do you think the IRS is wrong to look into the NAACP? Do you think that's a witch-hunt?

MFUME: I don't know. I know that it came at a peculiar time and I know that in 96 years of existence it has never happened before to the organization. I think just as 501c3s have an inherent responsibility to be careful they don't cross the line, I think the IRS has an inherent responsibility to be careful before instituting something like this because it can send the wrong messages.

Now, if you're absolutely certain you're going to prove your case that's one thing. But in this instance I think this will go on for a year or so, it will eat and drain the resources of the NAACP in defending it and getting papers read. But at the end of the day, I just don't believe it's going to stand. That's all. This was a speech that was copyrighted by Mr. Bond. Someone said earlier, yesterday, that if you're not allowed to criticize the president or any other institution of government, then what kind of society do we have?

And just the nature of the IRS saying, hey, we want to look at your finances and we want to look at this, we want to look at that on the basis of a complaint that we have yet to have seen, by the way, I think that's a little dangerous and I think it begs for caution, on both sides.

O'REILLY: But you've got to admit that Bond really was an anti-Bush guy and to get the tax exempt status, you're really supposed to be in the middle, you're not supposed to be favoring [one candidate over the other]. And the one thing that disturbed me [was that] we have been able to prove that the NAACP did hand out pro-Kerry literature at many of its meetings and I think rather than Bond's rhetoric, I think that that literature being handed out is going to hurt you.

MFUME: Do you mean at meetings of individual branches around the country, something like that occurred?

O'REILLY: No, the general headquarters and ...

MFUME: Oh, no, no. No, it didn't happen. I wouldn't allow that to happen on my watch. And I understand real clearly ...

O'REILLY: You can't know about everything.

MFUME: Yeah, but, Bill, I would be absolutely surprised if it did happen. We've been very, very careful about that in terms of the national operation. I can't say that it didn't, but I can tell you that it was an absolute— and remains— an absolute no-no. We just don't do that.

O'REILLY: Do you believe that African-Americans have a regressed under the Bush administration? Do you think they're worse off now, putting the economy aside because you've got to cut a little slack to any president who inherits a bubble recession and then gets hit on 9/11... I mean, it's just not fair. But do you believe that blacks have gone backward? And I just point out to the record homeownership among African Americans because I don't know. I don't know what the truth is. What do you believe?

MFUME: Well, I think the truth is somewhere in the middle. There's been real progress in some areas and you pointed to homeownership. There's been a real falling away in other areas like the resegregation of our public schools around this country, not some clandestine deliberate attempt to do that but when we look at the way schools are becoming populated by race, they are becoming more segregated. When you look at the tremendous increase and spread of poverty that we'd all like to think doesn't exist but we all know it does, the people who are poor have gotten poorer.

And now the question is, do you blame that on the president, do you blame it on the economy? Do you say that's not enough self-initiative? Is there something else? I think all of us share some of the blame, including organizations that are designed to work against those things. There has been some falling back. And I think if anybody looks at this with an honest eye they'll say, yeah, things have gotten bad in some areas and in some areas they may have gotten better.

O'REILLY: All right, Mr. Mfume, we wish you the best. You're a standup guy ...

MFUME: Thank you.

O'REILLY: ... and we look forward to having you on “The Factor” whenever you want to come on. Thanks very much.

MFUME: Well, thank you very much. I do appreciate it.

Content and Programming Copyright 2004 Fox News Network, L.L.C. ALL RIGHTS RESERVED. Transcription Copyright 2004 eMediaMillWorks, Inc. (f/k/a Federal Document Clearing House, Inc.), which takes sole responsibility for the accuracy of the transcription. ALL RIGHTS RESERVED. No license is granted to the user of this material except for the user's personal or internal use and, in such case, only one copy may be printed, nor shall user use any material for commercial purposes or in any fashion that may infringe upon Fox News Network, L.L.C.'s and eMediaMillWorks, Inc.'s copyrights or other proprietary rights or interests in the material. This is not a legal transcript for purposes of litigation.