Matthaus's comments comes after an 'inexperienced' German squad reached the semi-finals of this year's FIFA World Cup in South Africa before being eliminated by Spain.
"Germany have played better without Ballack and have had more success. Other players have taken the lead and there is a new hierarchy within the team," The Sun quoted Matthaus, as saying.
"I understand Ballack's ambition and desire to return but he should resign. Standing down would show true greatness, the realization the team is strong enough without him," he added.
He further said that Ballack's absence had helped the German team to reach the semi-finals of the World Cup.
"I don't mean that in a spiteful way, but he was holding up players who have now blossomed," he added.
Ballack was due to lead Germany in South Africa until he suffered damaged ankle ligaments while playing for Chelsea in May's FA Cup final.
World Cup: Rampant Germany march into last four
FIFA credit Dutch equaliser to Sneijder
Tata to head India-South Africa CEO forum
FIFA, WC organiser considering Vuvuzela ban
South Africa beat France to restore pride