Rajan, in a speech in Goa, said India needed to find a balance between giving government and regulators "unchecked power and creating complete paralysis".
Rajan did not cite specific examples, but his comments come as India's emboldened regulators have begun investigating companies over a variety of alleged misdemeanours.
Some investors have raised concerns about excessive zeal.
Budget 2015: Complete Coverage
The Central Bureau of Investigation sparked indignation after publicly announcing a probe into industrialist Kumar Mangalam Birla in December 2013 in relation to a coal block allocated to a firm belonging to his Aditya Birla Group.
It dropped the case in August, citing lack of evidence.
"Let me emphasise, we need 'checks and balance', but we should ensure a balance of checks. We cannot have escaped from the License Permit Raj only to end up in the Appellate Raj," Rajan said at an event in Panaji, Goa's capital.
"License Raj" has been used in India to describe what company executives say is a daunting array of government permits. Getting some of them has often involved complaints of demands for bribes.
Prime Minister Narendra Modi was overwhelmingly elected in May last year with a mandate of economic reform and speeding up decision-making, especially in areas such as infrastructure.
Rajan also said India needed more specialists within government and regulation as the economy develops.
"In many areas of government and regulation, as the economy develops, we need more specialists, with the domain knowledge and experience," he said.
"For instance, well-trained economists are at a premium throughout the government, and there are far too few Indian Economic Service officers to go around."
Shivaji revived Indian naval power
'Make in India for India makes more sense'
'The Gandhis never influenced environmental clearance for big projects'
Nearly every Indian household has a bank account
'GDP forecast is only adding to confusion'