Slovenia took a surprise 2-0 lead by half-time, outplaying the Americans and scoring with goals from Valter Birsa in the 13th minute and Zlatan Ljubijankic in the 42nd.
But Landon Donovan scored a brilliant individual goal three minutes after the break and as the Americans poured forward, Michael Bradley, son of coach Bob Bradley, grabbed an equaliser eight minutes from time.
Slovenia grabbed the lead through a superb 25 metre strike from Valter Birsa -- left in acres of space, the winger looked up and fired past a strangely stationary Tim Howard.
The US were struggling to match-up in midfield with Birsa causing isolated US left-back Carlos Bocanegra all kinds of trouble but they went close to levelling in the 41st minute when Clint Dempsey crossed low to Landon Donovan but Miso Brecko slid in to clear.
But a minute later, the US were caught on the break, Milivoje Novakovic split open a flat back line with a superb through ball to Zlatan Ljubijankic who kept his cool and slotted past the advancing Howard.
US coach Bob Bradley had plenty to sort out at the break and he brought on Maurice Edu and Benny Feilhaber to the midfield, taking off Jose Torres and striker Robbie Findley.
Within three minutes of the restart the US pulled a goal back, Landon Donovan cutting in from the right flank and then blasting in at the near post from the tightest of angles.
The US poured forward in search of an equaliser and eight minutes from time Michael Bradley surged into the Slovenia penalty area to get onto a knock-down from Jozy Altidore and put a rising shot past keeper Samir Handanovic.
Slovenia lead Group C with four points from the United States on two.
England, with one point so far, play Algeria, with none, later in Cape Town.
Earlier, Serbia pulled off a stunning 1-0 victory over strongly favoured Germany, who played most of the match with 10 men after striker Miroslav Klose was sent off.