CBI sources said as per the laid down procedure, the agency has taken over the investigation in the FIR registered by the local police which had named 11 persons in the case.
A team of agency officials reached the crime scene on Thursday and collected forensic evidence.
The team headed by a DIG level officer is in the process of understanding the circumstances of the incident.
In the wee hours of October 20, some upper caste persons had allegedly poured petrol from the window and set ablaze the house of Jitender, a dalit, in which his wife Rekha, 28, suffered serious burn injuries while Vaibhav, who was two-and-a-half years old, and his sister, 11-month-old Divya, perished in the fire.
Jitender, 31, also received injuries while trying to save them, the sources said.
The woman is still in critical condition in Safdarjang Hospital in Delhi.
The local police has registered a case against Sunped resident Balwant and his son Dharm Singh besides nine others under IPC sections 302 (murder), 323 (voluntarily causing hurt), 324 (Voluntarily causing hurt by dangerous weapons or means), 148 (Rioting, armed with deadly weapon) and 149 (Every member of unlawful assembly guilty of offence committed in prosecution of common object).
Others named in the FIR include Jagat, Edal, Naunihal, Joginder, Sooraj, Akash, Aman, Sanjay, and Desh Raj of the same village.
An SIT was constituted by the district administration under DCP PC Pawar.
As the incident sparked outrage in the country, Haryana Government decided to go for a CBI probe.
The CBI team was accompanied by DCP Pawar, ACP Vishnu Dayal and ACP Gajendra Singh.