Mathai, a 1974-batch IFS officer, will assume office from August one and will be there for a term of two years, official spokesperson Vishnu Prakash said.
He is serving as ambassador to France since January 2007. He is also concurrently accredited as Ambassador to the Principality of Monaco.
After completing post graduate studies in Political Science at the University of Poona, Mathai joined the Indian Foreign Service in 1974.
As Joint Secretary in the Ministry of External Affairs in New Delhi between January 1995 to February 1998, he headed the division dealing with India's relations with Bangladesh, Sri Lanka, Myanmar and Maldives.
He also served as Indian Ambassador to Israel from February 1998 to June 2001 and was the Indian Ambassador in Qatar from August 2001 to July 2005 and in Indian embassies in Vienna, Colombo, Washington, Tehran and Brussels.
He held the post of Deputy High Commissioner of India to the UK in London from August 2005 to January 2007.
Rao, who is retiring on July 31, has been appointed as India's ambassador to the United States.
India seeks details of MiG 29K crash from Russia
Awaiting Pak minister's July visit: India
Is Sonam Kapoor India's new gay icon?
Gandhi doesn't need this defence
Nirupama Rao to visit Pak by June end