Substitutes Marouane Fellaini and Dries Mertens turned the game around after Algeria playmaker Sofiane Feghouli had given them a 25th minute lead with a penalty at the Mineirao stadium.
Belgium, back at the World Cup after a 12-year absence, enjoyed the majority of possession but struggled to break down a disciplined two-line Algerian defence which kept Eden Hazard and forward Romelu Lukaku well in check.
The Algerians, with El Arabi Soudani surprisingly in the starting lineup for key forward Islam Slimani, posed little threat up front with winger Riyad Mahrez firing well wide in their only foray in the Belgian box early in the game.
They were far more successful the second time they ventured forward when left back Jan Vertonghen brought down Feghouli after a fine cross from Faouzi Ghoulam.
Feghouli calmly fired past keeper Thibaut Courtois to end his country's 28-year wait for a World Cup goal and send the green-white-clad Algerian fans delirious.
Belgium's Axel Witsel tried his luck with a couple of long-range shots but the Red Devils looked nothing like the menacing team that qualified unbeaten for the tournament.
Coach Marc Wilmots, the last Belgium player to score in a World Cup before Tuesday, took off lacklustre Lukaku early in the second half, bringing on teenager Divock Origi in the hope of breathing new life into his team.
It almost paid off instantly when the 19-year-old raced clear but failed to beat keeper Rais Mbohli with Algeria firmly on the backfoot.
It was Wilmots' other substitute, Fellaini, who delivered when he rose high to meet a Kevin De Bruyne cross and drill his header off the bar and into goal on 70 minutes.
The Desert Foxes, who had hardly put a foot wrong in defence, were caught napping 10 minutes later when Hazard raced down the left wing, and cut back perfectly for subsititute Mertens to fire in for the win.
Image: Dries Mertens of Belgium celebrates after scoring
Photograph: Jeff Gross/Getty Images
Belgium's pool of thrilling young talent is the envy of Europe
Europa League PHOTOS: Benfica hold Juventus, to meet Sevilla in final
Who will win the FIFA World Cup? Brazil, Argentina, Spain?
Football World Cup: How the 10 top teams stand a month before kick-off
How Bale wrote his name into Real Madrid folklore