He was administered the oath by Governor Ganga Prasad at the Paljor Stadium.
Golay is not a member of the state assembly at present as he did not contest the polls.
Hundreds of SKM supporters who were at the stadium cheered the 51-year-old party chief when he was taking the oath in Nepali language.
Former chief minister Pawan Kumar Chamling and senior Sikkim Democratic Front leaders were conspicuous by their absence at the swearing-in.
The SKM, founded in 2013, won a slender majority in the 32-member Sikkim Legislative assembly by bagging 17 seats against 15 won by the SDF.
The Chamling government was ousted by the SKM after over 24 years.
5-time CM Chamling loses in Sikkim
The Beauties of Sikkim
Pics: The Sikkim that made A R Rahman fall in love
PHOTOS: Why Sikkim is the best region in the world
Why do the Chinese want a train to Sikkim?