Kumble took three for 57 off 30 overs to move past Courtney Walsh's 519 wickets and help India to a big first innings lead of 373.
India's most experienced bowler, who will be 36 this October, is shouldering more responsibility in this series after the omission of off-spinner Harbhajan Singh from the first two Tests.
"I look on myself as a key member of the side. Having played for this long, the pressure is on every one and more so on me," Kumble told reporters. "As a senior bowler, it is important for me to lead the way. I'm happy that so far it's been so good."
India, who made a healthy 588 for eight declared in their first innings, dismissed the West Indies for 215 at tea on the third day. Forced to follow-on, the Caribbean side reached 43 for one in their second innings.
Fast bowler Munaf Patel and off-spinner Virender Sehwag assisted Kumble in the first-innings effort by also taking three wickets each.
With two days remaining, India are favoured to win the match, but they will want to remove West Indies captain Brian Lara quickly on the fourth day.
"He is obviously a key member for the West Indies," Kumble said. "Whether he comes in at number three or four, it is important for us to get his wicket early. Hopefully we should be able to do that."
Lara pushed himself up to number three in the second innings -- ahead of Ramnaresh Sarwan -- and was 15 not out at stumps.
