The sources said the agency team today reached Tihar Jail where Rajan is lodged after being deported from Indonesia and started questioning him after completing necessary formalities.
They said Rajan's questioning in the case is likely to continue for some days.
A special Maharashtra Control of Organised Crime Act court in Mumbai has granted the CBI permission to interrogate Rajan in the case.
Special Judge A L Pansare had allowed the CBI to question 54-year-old Rajan for 10 days starting January 27 before adjourning the case till February 5.
The murder of crime and investigation editor of MidDay is one of the prominent cases in which Rajan's hand is suspected.
The 56-year-old Dey was shot dead by motorcycle-borne shooters in June 2011, allegedly at the behest of Rajan. Four persons on two motorcycles fired at least four to five rounds at Dey, who was also riding a bike, from behind near SpectraBuilding at D Mart in Hiranandani area of Powai in Mumbai.
The police had said Rajan, who was upset with two articles written by Dey that purportedly portrayed him in "bad light", is suspected to have ordered his elimination. In 2012, the police had filed charge sheet against journalist Jigna Vora in the case.
According to the 3,055-page charge sheet, Vora allegedly instigated Rajan to execute Dey owing to professional rivalry.
Mumbai Police had filed charge sheet against Chhota Rajan and his aide Nayan Singh Bisht, shown as absconding, Rohee Thangappan Joseph alias Satish Kalya, Abhijeet Shinde, Arun Dake, Sachin Gaikwad, Anil Waghmode, Nilesh Shendge, Mangesh Agawane, Vinod Asrani, Paulson Joseph and Deepak Sisodia under the MCOCA in 2011.
Rajan is facing around 70 cases in Maharashtra, including the one related to the murder of Dey. Maharashtra government has handed over all the cases against him to the CBI.
J Dey murder case: CBI gets permission to quiz Chhota Rajan for 10 days
#SoCool: Mumbai police is a smash hit on Twitter
J Dey murder case: Spl court to start daily trial from Dec 17
Rajan's physical production in Maha court not feasible: Court
PHOTOS: Why Mumbai sleeps easy