The top Indian diplomat was received by officials of Sri Lanka's external affairs ministry and the Indian high commission at the Colombo airport, officials said.
The foreign secretary was expected to discuss the killings of two Indian fishermen mid-sea earlier in January. The incident had sparked a row between the two neighbours.
She is expected to meet President Mahinda Rajapksha on Monday to convey New Delhi's concern over the killing of fishermen allegedly by the Sri Lankan navy.
The government and the Lankan navy have already denied involvement in the death of the fishermen. Rao is also expected to hold talks with Lankan External Affairs Minister Professor G L Peiris.
Reflecting the political sensitivities in Tamil Nadu, India has already reacted very strongly to the incidents, in which two Indian fishermen were killed in the span of one month.
Rao is also expected to discuss measures to protect the Mahabodhi Centre in Chennai and Sri Lankan pilgrims to Budh Gaya.
Three Buddhist monks, including a Nepalese, were injured in an attack on the Sri Lankan Mahabodhi centre in Chennai last week.
"The two countries enjoy close bi-lateral ties and senior officials of Sri Lanka and India can meet at short notice whenever there are urgent matters to be discussed," said a senior Lankan official.
Lanka seeks humanitarian solution to fishing row
Third party behind attack on Indian fishermen: SL
Sri Lanka navy denies killing Indian fisherman