Govou lashed home the 66th minute winner after Robert Pires, captaining France for the first time, had converted a 43rd minute penalty before Shunsuke Nakamura deservedly levelled for Japan on the hour mark.
The win was marred only by a stoppage-time red card for France defender Willy Sagnol.
France, who beat Colombia 1-0 in their opener, have six points, while Japan and the Colombians, who beat 10-man New Zealand 3-1 earlier on Friday, both have three points before the final round of matches. The top two go into the semi-finals.
Japan, looking for revenge after their 1-0 defeat by France in the 2001 Confederations Cup final, played the better football in the first half and hit the woodwork in a spirited closing assault on the France goal at the Stade Geoffroy Guichard.
France coach Jacques Santini, missing his midfield trio of Zinedine Zidane, Claude Makelele and Patrick Vieira through club duty and injury, retained only midfielder Olivier Dacourt from the starting 11 of their laboured win over Colombia.
Fabien Barthez returned between the posts and midfielder Pires was given an unfamiliar role on the right flank.
SLOPPY PASSING
France, often let down by sloppy passing, failed to make any headway against a well-drilled Japanese defence in the first half and were given little or no room in the middle of the park.
At the other end, Japan's neat passing game, orchestrated by skipper Hidetoshi Nakata and Nakamura, was more impressive but consistently ran out of steam around the penalty area.
The 33,000 crowd, who did the Mexican wave to relieve the tedium, were finally roused by the first scoring chance in the 26th minute -- when Govou headed wide from a Steve Marlet cross.
But the flowing football was all coming from Japan and the crowd had just begun booing, whistling and calling for Thierry Henry to be sent on when calamity struck for Zico's Japan.
Junichi Inamoto, outstanding in the Japanese side's World Cup run on home soil last year, needlessly tugged the shirt of defender Jean-Alain Boumsong and Pires, Inamoto's former team mate during his spell at Arsenal, tucked home a low spot-kick.
France fared little better after the re-start and the fans' demands for pacy striker Henry became ever more insistent.
Japan were finally rewarded for their efforts when Nakamura, who scored twice in their 3-0 win over New Zealand, curled a free-kick in off the far post, leaving Barthez stranded.
Six minutes later, though, the hosts dug themselves out of trouble when Govou latched on to a long ball deep into the box and crashed home an unstoppable shot.
Japan shrugged off the setback and Nakata should have levelled minutes later, only to volley wide from close range, while Yasuhito Endo beat Barthez with another free-kick -- this time the ball hitting the bar and bouncing out.
Nakamura followed up with a superb shot but a deserved equaliser never came, while Henry finally arrived for France in the 80th minute and Sagnol was dismissed for a tackle on Nakata.
France round off their group games against New Zealand on Sunday in Paris when they will be joined in the semi-finals by the winners of Japan's match against Colombia in St Etienne.
