The home side were 204 for eight in their second innings at the close, still 24 runs away from making South Africa bat again.
South Africa made 330 in their first innings before Bangladesh replied with 102 and followed on 228 runs behind. The tourists won the first match in the two-Test series by an innings and 60 runs.
Bangladesh, who resumed their first innings on four for no wicket, batted without discipline and lost all their first innings wickets in the extended morning session.
The top order was unsettled by aggressive new-ball bowling from Shaun Pollock and Makhaya Ntini, and the first three wickets tumbled for 15 runs in four overs as the home side slumped to 37 for three.
Mehrab Hossain scored eight before edging a ball from Pollock to captain Graeme Smith at second slip.
Six balls later Javed Omar fell for 11 to a catch by substitute fielder Andrew Hall at first slip, earning Ntini his first wicket.
Habibul Bashar was given out leg before to Pollock for 14 despite the ball appearing to have struck his pad outside the line of off-stump.
Akram Khan endured a battering from Ntini before sparring at a delivery from the fast bowler and feathering a catch to wicketkeeper Mark Boucher to be caught behind for 13.
Alok Kapali was run out for one by Paul Adams before Mohammad Ashraful, on 15, edged a ball from Ntini to Pollock at second slip as Bangladesh stumbled to 66 for six.
Mohammad Salim was out for seven when he skied a delivery from Robin Peterson to Boucher, who took the catch at short mid-wicket to help the left-arm spinner take his first test wicket.
Medium-pacer Alan Dawson claimed his first test scalp when Mohammad Rafique edged to Pollock at second slip for a duck and also dismissed Tapas Baisya before the innings ended.
Ntini took three for 32 in 11 hostile overs, Pollock completed figures of two for 21 and captain Khaled Mahmud's unbeaten 20 was Bangladesh's top score.
Omar and Hossain made a steady start to their second innings before being dismissed in the space of eight balls.
Hossain was run out for 14 by Boeta Dippenaar's throw from the covers before Omar was well held by Pollock at slip for 27 off left-arm spinner Adams.
Bashar and Ashraful then shared Bangladesh's biggest partnership of the match, adding 47 runs for the third wicket before Ashraful, who scored 23, went to another Pollock slip catch, off Peterson.
Khan resumed hostilities with Ntini, pulling him for consecutive fours before driving the next delivery for another boundary.
However, Ntini had the last laugh when Akram fended the fourth ball of the over, a throat-bound snorter, to Jacques Rudolph at short leg for 23.
Bashar applied himself in a solid 33, but then cut expansively at a ball from Peterson and was caught by Boucher.
Mahmud fell for nought, Kapali, on 23, hoisted a ball from Dawson to Ntini at mid-wicket and Rafique ended the day's deluge of wickets with a rash heave at Adams that looped to Boucher.
Peterson was rewarded for his accuracy with a return of three for 46.
Scoreboard
South Africa (1st innings): 330 (Jacques Rudolph 71, Mark Boucher 71, Robin Peterson 61; Mohammad Rafique 6-77).
Bangladesh (1st innings): (overnight 4-0)
Javed Omar c sub (Hall) b Ntini 11
Mehrab Hossain c Smith b Pollock 8
Habibul Bashar lbw Pollock 14
Mohammad Ashraful c sub (Hall) b Ntini 15
Akram Khan c Boucher b Ntini 13
Alok Kapali run out 1
Khaled Mahmud not out 20
Mohammad Salim c Boucher b Peterson 7
Mohammad Rafique c Pollock b Dawson 0
Tapash Baisya b Dawson 4
Mashrafe Mortaza c Dippenaar b Peterson 1
Extras: (lb-4 nb-3 w-1) 8.
Total: (all out, 35.5 overs) 102
Fall of wickets: 1-22, 2-22, 3-37, 4-53, 5-62, 6-66, 7-73, 8-77, 9-85
Bowling: Pollock 8-3-21-2 (nb-3), Ntini 11-4-32-3 (w-1), Dawson 7-2-20-2, Peterson 8.5-1-22-2, Adams 1-0-3-0.
Bangladesh 2nd innings (following on)
J Omar c Pollock b Adams 27 (109)
M Hossain run out (Boucher) 14 (103)
H Bashar c Boucher b Peterson 33 (123)
M Ashraful c Pollock b Peterson 23 (73)
A Khan c Rudolph b Ntini 23 (21)
A Kapali c Ntini b Dawson 23 (52)
K Mahmud c sub b Peterson 0 (7)
M Salim not out 24 (58)
M Rafique c Boucher b Adams 18 (21)
T Baisya not out 8 (20)
Extras (b 5, w 1, nb 5) 11
Total: (8 wickets, 80 overs) 204
To Bat: Mashrafe Mortaza.
Fall of Wickets: 1-46, 2-46, 3-93, 4-119, 5-131 , 6-139 , 7-163, 8-190.
Bowling: Pollock 6-1-16-0 (2nb), Ntini 11-2-36-1 (3nb, 1w), Peterson 27-13-46-3, Dawson 10-5-12-1, Adams 19-3-70-2, Smith 7-0-19-0