The RBI cut the rate several times last year to reduce it by 125 basis points to the current 6.75 percent.
Panagariya also said in an interview that the RBI's main policy rate should be 50 basis points lower than it is now.
That puts him at odds with RBI Governor Raghuram Rajan, who is seen as unlikely to ease the repo rate further when the central bank reviews policy on February 2.
The RBI cut the rate several times last year to reduce it by 125 basis points to the current 6.75 percent.
While stressing he had an open mind, Panagariya said the time was right to review the RBI's target for consumer price inflation that sets a central goal of 4 percent, with a band of 2 percentage points either side.
He expressed particular concern over the lower end of the range, effectively 2 percent, saying this level was typical for advanced economies but unusual for emerging nations.
"I think we should go back to review the monetary policy framework," said Panagariya, appointed by Modi a year ago to run the Policy Commission, the government's main advisory body.
"The framework is fine, but the target needs to be revisited. It's been there for a year and we should review it."
If there is any revision to India's inflation target "it is not going to be downward", the former Columbia University economics professor added.
"It is premature to speculate what I would say. But I would go in with an open mind. If revisions have to happen, they will have to be upward," he said.
"Who has a floor of 2 percent among the developing countries? I have never heard of a developing country on a sustained basis having 2 percent or 3 percent inflation. That is the sort of target that most of developed countries have."
India formally adopted its monetary policy framework a year ago, shifting its focus to targeting consumer inflation from wholesale prices in a bid to subdue the country's historically volatile prices.
Asked about interest rates, Panagariya said that he had argued for a cut of 100 basis points before the RBI last eased rates by 50 basis points on September 29.
"My calculation is that 50 basis points still remains," he said.
Modi adviser Panagariya opposes 'tinkering' with fiscal deficit targets
Modi's action plan for start-ups: Up, up and away
Why Panagariya is worried over yuan devaluation
'I was hopeless in Maths'
Rate cut? Small savings schemes can still fetch you high returns