Rudd, an MP since 1998, said the "slings and arrows" of political life had taken its toll and he will leave Parliament at the end of this week.
"This has been the product of much soul-searching for us as a family over the last few months," the 55-year-old Labour leader said in an emotional speech to parliament.
"But for me, my family is everything, always has been, always will be, which is why I will not be continuing as a member of this parliament beyond this week. I wish Tony, his wife Margie and their family all the best for the rigours of high office that inevitably lie ahead," he was quoted by the Australian Broadcasting Corporation as saying.
Rudd, who served as the prime minister between December 2007 and June 2010, was again appointed to the top post in June this year in a last ditch attempt by the Labour party facing a sure defeat under the leadership of Julia Gillard. He was elected prime minister in the 2007 general election in a landslide defeat of John Howard, but within his first term he was dumped by colleagues fed up with his style of management and demoted to foreign minister, a post he served between September 2010 and February 2012.
Rudd's arch-foe and current Prime Minister Abbott described his retirement as "a special occasion".
"This is a significant moment in the life of this parliament, Abbott said and called him "one of the big figures. I salute him and I wish him and his family all the best for the future," he said.
Rudd said he planned to support causes for homelessness and organ donation and establish a "national apology foundation" to aid indigenous Australians in future.
Row in Australia over PM Gillard's 'small breasts' menu
Kevin Rudd ousts Aus PM Gillard as head of Labor Party
After 36 pc salary cut, Singapore PM STILL world's best paid
BIG BUCKS: Salaries that world leaders take home
In tiny Brunei, India makes inroads into ASEAN