Rawat, 65, was administered the oath of office by Governor Aziz Qureshi at Raj Bhawan lawns along with a 11-member Cabinet all of whom were ministers in the Bahuguna government.
Rawat's name was approved by Congress president Sonia Gandhi, senior party leader Janardhan Dwivedi told reporters after a five-hour-long meeting of the Congress Legislature Party. Rawat was Union Cabinet Minister for Water Resources.
His appointment comes at a time when Congress is trying to salvage its image in the hill state.
Bahuguna, who resigned on Friday, has drawn intense criticism for the way he handled last year's devastating floods and relief and rescue work.
In the 70-member assembly, Congress has 33 members and has the support of seven-member PDF including three BSP, one Uttarakhand Kranti Dal and three Independent MLAs. BJP has 30 MLAs.
The Progressive Democratic Front, which has four ministers in the outgoing Bahuguna ministry, has pressed for additional ministerial berths in the new government.
Rawat pipped two other front runners for the top post -senior party leader Indira Hridayesh and member of Bahuguna's cabinet Pritam Singh.
Spelling out his priorities, Rawat said it would be definitely to pull the state out of the fallout of the 2013 catastrophic flashfloods.
Besides Dwivedi, senior Congress leaders Ambika Soni and Ghulam Nabi Azad attended the CLP meet as central observers.
Having the reputation of a hard-working leader, Rawat, MP from Haridwar, was made a Minister of State for Labour and Employment in 2009 and then MoS for Agriculture and Food Processing Industries. In July 2011, he was made MoS for Parliamentary Affairs.
He was elevated to the rank of a cabinet minister in 2012 in an apparent bid to placate him after Bahuguna was made the CM.
Rawat has earlier served as a member of the Rajya Sabha from 2002-08. He won his first Lok Sabha election in 1980 from Almora by defeating BJP leader Murli Manohar Joshi. He won the Lok Sabha polls in 1984 and 1989 as well.
After Sonia's nod, Harish Rawat chosen CLP leader in Uttarakhand
Uttarakhand CM Vijay Bahuguna quits
Seemandhra Cong MPs upset over resolution issue at AICC meet
How the dynasty was kept alive at the AICC session
Congress draws up Rajya Sabha list with eye on Lok Sabha poll