Rakesh, aged 39, was undergoing treatment at Antwerp University Hospital in Brussels, where he was rushed on Tuesday after he developed sudden pancreas-related complications.
He had been on a European tour with his friends since last week.
Rakesh, the eldest son of Siddaramaiah, who had an acute pancreatic ailment for long, was critical but stable before his condition worsened, the sources said.
"Rakesh died of multi-organ failure," the Chief Minister's Office said.
Siddaramaiah, his wife and second son Yathindra Siddaramaiah, a doctor, and the family doctor were in Brussels to be with Rakesh.
He is survived by his wife, a son and daughter.
He had met with an accident 15 years ago, when he suffered injuries to his pancreas.
Soon after his son was hospitalised, Siddaramaiah had spoken to External Affairs Minister Sushma Swaraj, seeking her assistance in getting the best treatment for his son in Belgium.
Meanwhile, Prime Minister Narendra Modi has condoled the death of Siddaramaiah's son.
"My thoughts are with Siddaramaiah ji & his family on the demise of his son. May God give the family strength to bear the irreparable loss," he wrote on twitter.
Over 100 days later, no sign of 'Jai'
After Gurugram, now Delhi hits a gridlock
Blunders won't happen again, says Gurugram police
One feels sorry for Subramanian Swamy
Infiltration bid foiled in Kupwara, 2 terrorists, 2 soldiers killed