The world champions struggled to break down an inspired Georgian defence until Jaba Kankava was sent off by English referee Mike Riley 15 minutes after halftime for a second yellow card with the score at 1-1 following two early goals.
Midfielder Mauro Camoranesi headed the visitors into the lead within a minute of the red card after a fine cross from Antonio Di Natale and Simone Perrotta added a third from close range shortly afterwards in the Group B match.
Italy coach Roberto Donadoni took the victory in his stride.
"We knew coming here that it would be a tough game. Even when they equalised we stayed calm and never had any doubts that we would win in the end," he told a news conference.
His Georgian counterpart Klaus Toppmoeller blamed the red card for his team's defeat.
"I told the referee after the match that Kankava's foul didn't warrant even a yellow card," the German told reporters.
"But I'm happy with our play. We were on equal terms with such a strong team like Italy until that red card. We also controlled the game until their first goal and we were unlucky not to score the second late in the first half."
KEEPER FOOLED
Daniele De Rossi had opened the scoring in the 16th minute when his low drive from 25
Giorgi Shashiashvili equalised for the Georgians eight minutes later, sending most of the 55,000 crowd wild.
Shashiashvili beat Italy keeper Gianluigi Buffon with a curving shot into the top corner from just inside the area.
The Italians piled on the pressure late in the first half with Luca Toni wasting two good chances within a minute.
First, the striker fired a right-foot volley on the turn wide of the left post, then he just missed the far corner with a low shot after being set up by Gianluca Zambrotta.
The visitors, however, were lucky not to concede a second goal just before the break when defender Zurab Khizanishvili struck the post with Buffon well beaten.
Italy raised the tempo after the interval but the home team defended well until Kankava received his marching orders for tackling Andrea Pirlo from behind.
After a slow start when they managed only one point from their opening two games, the Italians now have seven points from four following a 2-0 win over Ukraine last Saturday.
Georgia, on the other hand, slumped to a third consecutive defeat after thrashing the Faroe Islands 6-0 in their opener two months ago. They have also lost to France and Ukraine.