The 70-year-old Ansari was administered the oath of office by President Pratibha Patil in the presence of Prime Minister Manmohan Singh and other top leaders at Rashtrapati Bhawan in New Delhi.
He will assume office as the Chairman of the Rajya Sabha on Monday.
Scholar-diplomat Ansari was elected Vice President yesterday. The UPA-Left nominee bagged 455 votes in the electoral college of 788, defeating NDA candidate Najma Heptullah by a big margin of 233 votes.
He succeeds Bhairon Singh Shekawat, who quit office after his defeat in the Presidential election on July 21.
Shekhawat was among those present during the swearing-in ceremony at the Ashoka Hall.
Congress President Sonia Gandhi, several Cabinet ministers and Leader of the Opposition L K Advani, also attended the function.