In a brief statement by way of clarification, Shinde, who is also the Leader of the Lok Sabha, said his comment has created a "misunderstanding" and that he had no intention to link terrorism with any religion.
"Since controversy has been created on account of my statement, I am issuing this clarification and expressing regret to those who felt hurt by my statement," he said.
"My statement given in Jaipur last month has created a misunderstanding. It has been understood to mean that I was linking terrorism to a particular religion and was accusing certain political organisations of being involved in organising terror camps.
"I had no intention to link terror to any religion. There is no basis for suggesting that terror can be linked to organisations mentioned in my brief speech in Jaipur," he added.
Shortly after Shinde expressed regret, BJP welcomed it but said it ought to have come much earlier.
"The delay had created a condition which gladdened the hearts of terrorists and their patrons in Pakistan," BJP chief spokesman Ravi Shanker Prasad said.
Shinde had made the controversial remark at Jaipur exactly a month ago at the Congress' Chinthan Shivir raising the hackles of the principal Opposition party and the Sangh Parivar.
Earlier in the day, Shinde along with Parliamentary Affairs Minister Kamal Nath had a meeting with Leader of the Opposition in Lok Sabha and BJP leader Sushma Swaraj after which government managers worked overtime to formulate a statement which was acceptable to both sides.
The Home Minister has been under severe attack from BJP which had promised to rake up the issue in the Budget session beginning on Thursday.
"I will continue to perform my duties to the best of my ability to ensure that harmony is maintained in the social fabric of India," Shinde said.
Shinde's statement came on a day when BJP took to the streets in Delhi to protest against his remarks.
Party spokesperson Nirmala Sitharaman said Shinde's remark gave a lot of "undue advantage" to terrorists and handlers of terrorism and had in a way "compromised on the national security".
She said the remark was "without any basis then and today in his statement it is conceded that the RSS and the BJP do not train terrorists in their camps.
"This whole month unfortunately has been spent on an issue, which had really created such a sense of worry and anguish among people, nationalist organisations and the BJP, which is the principal Opposition party."
She said terrorists or terrorism cannot have a colour and cannot be related to a religion.
"In fact the statement, which has come from the home minister should be seen as a lesson for many people, who every then and now throw this kind of baseless allegation against BJP and RSS as such allegations are only going to compromise with the national security or even the social harmony in the country," the BJP spokesperson said.
She hoped that with the air over Shinde's statement now being cleared, attention will be given to critical issues, which are pending in Parliament.