Krishna has resigned from the Congress and Congress Working Committee.
The Congressman is said to have written a letter to party president Sonia Gandhi apprising her of his decision.
The 84-year-old first became a Member of Parliament in 1968 from Mandya.
Krishna led the Congress to victory in Karnataka in 1999 and served as the chief minister till 2004.
He also served as the governor of Maharashtra for the period 2004-2008.
Krishna was also made the minister for external affairs in Dr Manmohan Singh’s cabinet, and resigned from it in 2012.