Rai's remarks about the alleged inaction of the former PM during various scams have generated sharp reactions from Congress party leaders, while other political parties have used his observations to hit back at the previous United Progressive Alliance regime.
The former CAG said that Dr Singh should also get credit for taking prompt action with regard to ordering of coal block auction, after certain irregularities were brought to his notice.
"The prime minister is the head of the government and he may not have any role in the decisions which some of them, like the Commonwealth Games he was not directly responsible for any thing.
"But ultimately the acclaim and the blame, both have to be taken by the head of the institution. In that respect, I have brought it out and nothing personal," Rai told PTI in an interview. "It is certainly not to do with an individual," he said.
Giving an example, the former CAG said, "Our Indian team went to England to play cricket and in all the Test matches we performed very poorly. Didn't the blame of it come to a very large measure on the captain of the team? May be a large number of players did not function or play properly, but the blame had to be taken by the captain."
Rai further said, "If I am the CAG today and if something goes wrong in my organisation, even if I did not have a role to play in it, the responsibility revolves around me. I give you the classic example that six, eight months back, when the navy had some accidents, the chief resigned on moral responsibility but he did not have any role to play in those accidents but the responsibility revolved around him," he said.
Rai's new book, The Diary of the Nation's Conscience Keeper -- Not Just an Accountant, talks about various issues observed during his 2008-2013 tenure as the CAG, including on Dr Singh's role at times like 2G and coal block allocations.
Reacting to Rai's observations, Dr Singh said on Sunday, "I, indeed, did my duty. Well, I do not want to comment on other people what they have written."
Asked for his comments on this, Rai told PTI, "That was his way of interpreting duty and it is perfectly fine because no two individuals would react similarly in a particular situation. If you were sitting on that seat, you would have had a different interpretation. That is perfectly natural."
Image: Former PM Dr Manmohan Singh
Former CAG says Manmohan Singh can't shirk responsibility in 2G scam
Manmohan Singh refuses to comment on ex-CAG's criticism
Manmohan has left behind no legacy: Natwar Singh
When PM Manmohan Singh almost quit