The Swedes lost the first game 11-8, but then reeled off four consecutive wins 11-6 11-9 11-9 12-10 to advance.
"We knew we had a good chance to win the match. We played against them in the past and it was very close, just a few points decided it," said the 38-year-old Waldner, a gold medallists in the singles at the Barcelona Games.
Kong, a double gold medallist, and Wang, a newcomer to the Games, looked to be in control, never trailing in the first game. But in the second they could not solve the steady play of their opponents.
The third game was much tighter and with the score level at 9-9 before the Swedish pair took the final two points.
Two more tight games followed, but the Chinese failed to find a way past and a couple of lucky breaks helped the Swedes close out the fifth game to take the match.
"The Swedes had a better backhand play than us today, and we also didn't play as fast and ferocious as our opponents did," said the 21-year-old Wang.
"And the Swedes were lucky at some crucial points in the last game," he added.
Wang said he would now concentrate on his singles matches. He faces Ju Lin later on Wednesday in a fourth round tilt. Waldner also competes in the singles later in the day, taking on China's Ma Lin, the current world number two.