The fast bowler broke down with a strain in his right groin during the fifth day of the second Test in Adelaide on Tuesday.
"I knew I had a chance of not playing but once you find out you won't play it hits you harder than you realise," said a disappointed Gillespie in a CA statement.
Australia, who trail India 1-0 in the four-match series, are expected to recall fiery paceman Brett Lee from injury on Monday when they name their team for the third test in Melbourne starting on December 26.
The
"It's now a case of getting things right for the rest of the summer as we have plenty of cricket left to play," said South Australian Gillespie, who has led the attack in the absence of New South Wales pair McGrath and Lee.
Australia team physiotherapist Errol Alcott said: "Jason has strained his right groin and although the medical staff is confident it will resolve over the short term, we felt it was prudent for him to miss this next Test match."
The fourth Test in Sydney starts on January 2.