The protesters reached gate no 3 of the building around 11:45 am and started shouting anti-China slogans.
The police tried to pacify them but they did not relent and eventually clashed with the police as they were stopped from approaching the gate. Some of the protesters were also detained by the police.
A group of Tibetans also held a protest at Dhaula Kuan and were forcefully removed from the spot.
On Wednesday, 10 Tibetans were detained after they staged a protest outside Chinese embassy against the visit of the Chinese president.
Xi is on a three-day visit to India.
Image: Protesters and police have a scuffle outside Hyderabad House, the venue where the protesters were agitating against Xi's visit. Photograph: ANI/Twitter
Prez Xi Jinping receives guard of honour at Rashtrapati Bhavan
PM raises issue of incursions with Chinese President Xi
Xi must rein in the PLA from running its own foreign policy
Tailored kitchen, special staff for China's first couple
Hope to take ties to a higher plane, says Chinese President Xi