Addressing a meet-the-press programme in New Delhi, he said his removal was a "punishment" for his honesty and integrity.
"I am first of all trying to know what are the causes (behind my removal) ...I honestly feel that it was a punishment for my honesty, integrity...I feel that I am not a fit person in this system," he said.
Asked whether he was going to file an RTI against the matter, he said everything was under consideration.
Kataria trained his guns at the present chief minister of Puducherry, saying "his first job was to get me removed because I was inconvenient".
Kataria said he has sought a meeting with the President of India to know the reasons for his removal.
He alleged his "firm stand against corruption and efforts to provide fair and transparent government" was what led to his ouster.
He said "a more fundamental question arises. What is the position of a Governor who is a constitutional head? It is a violation of the constitution.”
"It's a bigger question which is to be debated and considered and a way has to be found. How Governors are being treated? Overnight, I was removed. Overnight, Kamla Beniwal was shifted to North East and so many others... It is all petty politics", Kataria said.
He said "in Puducherry, I have seen the dirtiest politics, petty politics. It is not prevalent even in municipal committee and panchayat politics".
He said he had met the Prime Minister and the Home Minister and "I told them everything... I told them you say you are for a new system, you are a fair, transparent govt, zero tolerence for corrupt... You may remove me I don't mind because I am a Congressman.
Asked whether his removal had anything to do with his approval for filing of an appeal against the acquittal of Kanchi Shankaracharya, he said the approval was "granted on the recommendation of the Law secretary, Chief secretary and the Chief minister" and he signed it in a "routine way".
"I do not go into the file. There was no mention of Shankaracharya at all. When it came to me signed by everybody in the government, so I signed. I was never told. In a routine way, there are so many cases where a man is acquitted and the appeal is filed," Kataria said.
Former Arsenal star Ljungberg confirms ISL participation
EU, Australia-led group unite against India at WTO
I was a hindrance in Puducherry govt's way to plunder: Ousted governor
High speed trains planned in these 9 routes
Transfer of Governor Beniwal vindictive, alleges Vaghela