The CM went to famous Chhoti Patan Devi and Badi Patan Devi in Patna city in the morning and also visited several other places in Patna and sought blessings for peace and prosperity of the state, officials in the CM's office said.
He is scheduled to visit many more puja pandals later in the day, they said.
Deputy Chief Minister Sushil Kumar Modi also visited the pandals.
"Every year I make it a point to visit puja pandals in Bangali akhara and Ramkrishna Mission among others," Modi, who spent his childhood in Musallahpur locality of the town, told PTI.
He said he also keeps 'navratra vrata', which is celebrated for nine days to mark the vanquishing of the demon.
Modi's colleague and Health Minister Ashwani Choubey is also deeply involved with puja.
Bihar Minister for Art, Culture and Youth Affairs Sukhda Pandey has arranged for a 'puja kalash' at her home and recites hymns for long hours.
"I do puja and if given time go around different puja pandals in the town," Pandey added.
Her colleague and Industry minister Renu Kumari said she will keep moving around in different puja pandals in Madhepura district till October 25.
Nitish to lead Bihar delegation to Pakistan in Nov
Thackeray praises Modi for barring Bihari leaders
Nitish, Modi appeal for calm in Madhubani district
BJP snubs Nitish, to contest 40 LS seats in Bihar
'Education drive needed before GST is introduced in India'