At the same time, Tharoor was critical of the Modi government, saying there is a huge gap between rhetoric and reality.
Tharoor, whose wife Sunanda Pushkar's mysterious death has led police to register a murder case and triggered a controversy, also lashed out at the media, alleging it is "irresponsible".
"I didn't expect him (Modi) to reach out and that's why I was very impressed," the Congress MP said during a media interaction at the launch of his book India Shastra in the Apeejay Kolkata Literary Festival.
"I was surprised because Modi had not shown signs of reconciliation with an opponent before that. He and I had also clashed quite clearly and explicitly 3 to 8 months previously when he had said something about my wife," he said, recalling Modi's "nasty" tweet about Sunanda.
In an apparent attack on Pushkar, Modi had said at a Lok Sabha election rally in Shimla last year, "Wah kya girlfriend hai. Aapne kabhi dekha hai 50 crore ki girlfriend? (What a girlfriend? Have you ever seen a 50-crore girlfriend?)”
Tharoor had later taken on Modi by tweeting, "My wife is worth a lot more than your imaginary 50 crore. She is priceless."
Tharoor, who had accepted the prime minister's invitation to join the 'Swachh Bharat' mission of the National Democratic Alliance government, said, "I must admit that I was taken aback. It was a very gracious thing to do".
Tharoor recalled that Modi had also reached out to him earlier on Twitter but that had political reasons as he had appealed to him to get young voters to register.
“I joined because that was the cause I believed in," he said.
Tharoor did not take any questions from the media when asked about the Sunanda murder case.
He also praised BJP for including more women in the party, terming it as a "healthy sign".
“I am pleased that the BJP is finally making room for the right gender because that was a very, very male dominant government and a male dominant party. So that's a healthy sign."
In a no-holds-barred attack on the media, Tharoor said that the advent of 24/7 TV channels and the pressure of breaking
"I have been misquoted in media, being defamed now day in and day out," he said, adding, the media is "free and irresponsible".
"90 per cent of Indian journalists do not record what I am saying. Almost every quote attributed to me in the Indian media is an inadequate para-phrase by the journalist of what he thinks I said.”
"My fluency is only valued to those who are actually hearing me. If tomorrow there is a newspaper article about this, I guarantee the quotes will be wrong".
Tharoor also said that there was always either a by-poll somewhere or state assembly polls and to win the elections Modi has to take the help of RSS to "polarise the electorate".
He said that Modi himself started off as an RSS pracharak and so he also belongs to the same school of thought.
Taking a dig at the moral brigade, Tharoor said he is lucky that his 2013 book The Great Indian Novel was not banned as the "mounting intolerance" noticed now was absent 25 years ago.
"I was lucky the book didn't get banned and thereafter, I haven't looked back," Tharoor said.
Tharoor expressed confidence that the book would not have survived in the present climate of "mounting intolerance that we seem to be living with in our country."
"The Great Indian Novel was one of the first of its kind in India 25 years ago that poked fun at both the hallowed heroes of the nationalist movement and also timeless legends of the great legend Mahabharata," he said.
The best-selling author of books like Midnight to the Millennium and the Elephant, Tiger and Cellphone recalled that Khushwant Singh had devoted an entire column to this book but with a warning that 'Go out and buy this book immediately because it will be banned within a week'.
But Tharoor could get way with some "very minor changes" to the manuscript.
"I don't know what saved me. Had Indians suddenly become more mature or was it simply because the kind of people who banned books were not trying to read English language books in 1989," the award-winning writer said.
When a book-lover asked him about how he prefers to write -- with pen and paper or on the computer -- Tharoor said he composes directly on word processor on the computer.
He joked about his hand-writing saying his hand-written stories look like a doctor's prescription.
Army's all-women contingent marches into history
'Tharoor told me to persuade Sunanda to be normal'
Tharoor knows who murdered Sunanda, says Swamy
Trade gap at 10-month low on falling oil, weak gold demand
Vibrant Gujarat: Do the numbers add up?