At stumps on day four, India were struggling at 53 for three in 18 overs after being set a stiff 257 for victory, on a pitch offering assistance to the spinners.
Their hopes now rest on the in-form Sachin Tendulkar, who was unbeaten at stumps on 11, with nightwatchman Ishant Sharma was not out on two.
Off-spinner Randiv, who took the new ball, was Sri Lanka's hero, claiming all the three Indian wickets to fall, for magnificent figures of three for 20 in nine overs.
Virender Sehwag was caught at first slip by Mahela Jayawardene off the third ball he faced and became Randiv's victim for the third innings in succession. Rahul Dravid played one back to his stumps for seven, while Murali Vijay was controversially given caught at leg slip for 27.
Earlier, half-centuries from Thilan Samaraweera and Ajantha Mendis helped Sri Lanka to post a fighting 267 in 85.2 overs.
Mendis, who hit his maiden fifty in Tests, played a vital knock of 78 from 157 balls before he was caught in the covers off Amit Mishra.
Samaraweera, who hit a century in the first innings, again came to the team's rescue. He scored 83 from 138 balls, putting on 118 for the ninth wicket with Mendis to register Sri Lanka's best ninth-wicket partnership in Tests and helping them comeback after they were reeling 87 for seven at one stage.
All the three Indian spinners -- Pragyan Ojha, Amit Mishra and Virender Sehwag -- made most of the conditions, claiming three wickets each. Ojha was the pick of the bowlers, taking three for 89 in 89, to follow up on the four for 115 he took in the first innings.
Part-timer Sehwag made an important contribution with the ball, claiming three for 51 to go with his quickfire century (109) with the bat, while Mishra took three for 47.
Sri Lanka lead the series 1-0 after winning the first Test by 10 wickets, while the second ended in a high-scoring, dull draw.
Morning session: (28 overs, 98 runs, 6 wickets)
Ojha started proceedings on day four and immediately made an impact in his first over. Night-watchman Suraj Randiv survived a close leg before appeal, while Sangakkara edged one through the vacant slip region.
Randiv was dropped by Dhoni when he edged one that bounced and spun sharply, but Ojha had his man off the very next delivery. He beat his defensive shot with a slightly quicker delivery that struck him in front of the stumps and trapped him leg before wicket for six.
Ishant Sharma bowled his heart out in the morning session, bending his back and using the short ball to good effect, and even hit Sangakkara once on the chest.
Ojha then killed off Sri Lanka's hopes with the wickets of Mahela Jayawardene and Kumar Sangakkara in successive overs.
The left-arm spinner got the wicket of Mahela Jayawardene -- the batsman edged one that turned sharply and was caught at first slip for five.
Sangakkara again gifted his wicket away to Ojha. Sri Lanka's captain pulled a short delivery straight to Raina at square leg and was dismissed for 28, inclusive of a boundary in 49 balls.
Angelo Mathews looked to hit Sri Lanka out of trouble and charged down the wicket to Ojha and hit him down the ground for a four, in the 31st over.
Mishra also got among the wickets early on as he took two in his second over. First Mathews (5) hit a full toss straight into the arms of Sachin Tendulkar at midwicket and Prasanna Jayawardene was trapped leg before wicket off the next delivery.
Sri Lanka were staring at the inevitable, reeling at 87 for seven in the 32nd over, having lost their last five wickets for 24 runs in the space of 11 overs.
Lasith Malinga looked to counter-attack and heaved Ojha over midwicket for a boundary. Thilan Samaraweera also realised the need for quick runs and hit a boundary each in back-to-back overs off the spinners to take the total past the 100-run mark in the 35th over.
Just when the Malinga-Samaraweera partnership was beginning to frustrate the Indians, it was Sehwag who did the trick yet again.
The off-spinner got his third wicket, trapping Malinga leg before with a quicker delivery for 15, but not before the pacer had put on 38 vital runs for the eighth wicket.
Mendis also looked to attack and score few runs as he slammed Sehwag over cover for a boundary, while Samaraweera swept the last ball of the over for the same result.
Mendis ended the session in style when cut a short delivery from Ishant through point for a boundary.
At lunch on day four, Sri Lanka were precariously placed at 143 for eight in 44 overs, for an overall lead of 132 runs.
Ojha bowled a wonderful spell in the morning session to start the rot as he took three for 58 in 17 overs. The striking aspect of Ojha's bowling once again was the fantastic control he showed with his line and length and was deservedly rewarded with three wickets.
Sehwag continued to torment the Sri Lankans, this time with the ball, as he took three for 26 in seven overs, while Mishra claimed two vital wickets.
Post-lunch session: (30 overs, 82 runs, 0 wicket)
If the morning session belonged to India, the post-lunch session was all Sri Lanka, as they milked the bowlers around, while not losing a single wicket in the process.
Samaraweera got the first boundary after the break when he chipped down the wicket to Mishra and lifted it down the ground, in the 48th over.
Even Mendis grew in confidence as he defended resolutely while taking the singles at regular intervals to take off the some pressure from Samaraweera.
With the partnership growing, the Indians got impatient and soon Sehwag was introduced to break the stand.
Samaraweera swept Sehwag for a cracking boundary through the leg side to bring up his fifty from 79 balls, in the 55th over.
Mendis also continued to blossom as he drove a full delivery from Sehwag for a delightful boundary through the covers.
India brought back Ishant to pepper Mendis with the short deliveries but Mendis played the uppercut over the slips for a four. However, in his next over, Ishant caused Mendis some pain when he hit him on his fingers with a fast bouncer directed on his body.
An angry Mendis then slog-swept Ojha over square leg for a six in the next over to move to 38 and register his highest Test score, he best being 27.
Captain Dhoni rotated his bowlers around regularly but it had no effect on Mendis and Samaraweera, who took Sri Lanka to 211 for eight in 68 overs for a 200-run lead.
The two batsmen put on 100 runs for the ninth wicket in 190 balls as the Indian bowlers were really clueless and short of ideas against the duo.
At tea on the fourth day, Sri Lanka were 225 for eight in 74 overs, an overall lead of 214.
Samaraweera was instrumental in Sri Lanka's recovery with superb rearguard action of 78 from 119 balls, inclusive of six boundaries. Mendis also made a vital contribution with the bat, scoring 45 from 114 balls, inclusive of four boundaries and a six, as the two batsmen put on 100 runs for the ninth wicket.
Post-tea session: (29.2 overs, 95 runs, 5 wickets)
India employed a strange field for Ishant bowling round the wicket to Mendis, with two slips and three men close to each other in the third-man region, and just one man in the covers.
Mendis completed his half-century from 122 balls in some style when he charged down the wicket to Ojha and lifted him over cover for a boundary, in the 77th over.
In fact, Mendis batted like a proper top order batsman, as he waited for the loose balls while defending the good ones. He took on a full delivery from Mishra and lifted him over the mid-on fielder for a boundary, his sixth of the innings.
Sri Lanka were 241 for eight in 80 overs before India took the second new ball, hoping for things to change.
The move worked as Mithun struck in the first over with the new ball, claiming the key wicket of Samaraweera. The right-hander, who made 83 from 139 balls, tried to pull a short delivery but got an edge and was caught down the leg side by wicketkeeper Dhoni.
It was the Malinga-Samaraweera partnership that gave Sri Lanka hope; they put on 118 runs in 233 balls for the ninth wicket, Sri Lanka's highest for that wicket in Tests.
Mendis decided it was time to cut loose with just one wicket left. He attacked Ishant and got two streaky boundaries behind the wickets, but the third one came right off the middle of the bat over mid-on.
Just like the first Test in Galle, India's bowlers were found wanting against the tailenders once they went on the attack.
Mendis unleashed another amazing lifted shot over the covers in Ishant's next over for his tenth boundary of the innings that took his score to 76.
Dhoni immediately brought on the spinners and it was Mishra who ended the innings when he had Mendis caught in the covers.
Sri Lanka were dismissed for 267 in 85.2 overs, setting India 256 for victory that would level the three-match series.
Mendis hit a career-best 78 to register the highest score for Sri Lanka by a number ten batsman in Tests. The spinner hit 10 boundaries and a six in his 157-ball knock, playing a big role in Sri Lanka's revival which had seemed impossible at one stage.
For India, the spinners did most of the damage but must be ruing having let their opponents get away at the end. Ojha excelled with figures of three for 89 in 28 overs, and Sehwag played a crucial role by claiming three for 51 in 15 overs.
Mishra found his rhythm in the second innings to take three for 47, while Ishant bowled a great spell without any success.
(India innings)
Murali Vijay gave India a good start with two boundaries in the opening over from Malinga.
Surprisingly, Sri Lanka gave the new ball to Suraj Randiv ahead of Chanaka Welegedara, who went for a lot of runs in the first innings. What may have also counted in Randiv's favour was that he had dismissed Sehwag in the last two innings.
And the move worked, with Randiv getting the prized wicket of the in-form Sehwag in his first over. The right-hander opened the face of the bat and guided one straight to Mahela Jayawardene at first slip to fall for a three-ball duck.
Rahul Dravid also didn't survive long; he played one back on to his stumps and was bowled by Randiv for seven as India slipped to 33 for two after ten overs.
It turned out be a forgettable series for Dravid as he managed just 95 runs in the three matches at an average of 19 with a highest score of 44.
Sachin Tendulkar got his first boundary in streaky fashion when he edged one wide of the slips off Chanaka Welegedara.
Vijay survived a close leg before shout against Randiv when he was struck right in front of the stumps but umpire Simon Taufel turned down the decision, in the 14th over.
But Randiv got his man in the next over. Vijay was controversially given caught at leg slip for 27. Australian umpire Taufel consulted the third umpire before ruling in favour of the fielder despite replays not being conclusive after Mahela Jayawardene appeared to have taken one very close to the ground.
At close on day four, India were 53 for three in 18 overs, still needing another 204 runs for victory. Tendulkar was unbeaten on 11 from 22 balls, while nightwatchman Ishant was on two.
Randiv was superb with the ball, especially from around the wicket, as he claimed three for 20 in nine overs.