Playing her first tournament since a groin strain forced her to pull out of the Family Circle Cup in Charleston last month, Williams struggled to find her rhythm against the 14th seed and paid the price for an error-strewn performance.
"I'm just really surprised that I won. In the first set she was missing a lot, then in the second my game went wrong," said Schnyder.
"The third set was just really tough, very close. I tried to mix it up, but she's such a champion that you have to expect her to hit a lot of winners. I'm really happy and lucky to be through to the next round."
Schnyder's reward will be a semi-final against third seed Jelena Jankovic, who produced an impressive display of baseline hitting to sweep past Elena Dementieva 6-2 6-1.
Williams, the Australian Open champion, was left to reflect on a wayward performance.
"I just didn't make the shots that I should have. I probably hit about 50 unforced errors, which is pretty good to get to 7-6 in the third (set)," she said.
Asked whether she still felt confident of a good run at Roland Garros later this month, the American replied: "Absolutely. It's good to get some hard matches and long points under my belt.
"I think not winning today is going to work well for me because it will give me time to get even fitter. As soon as I start to believe in myself I'm going to be a lot better."
Schnyder had lost six of her previous seven matches against Williams, though her one victory had come on clay, at Charleston in 2002.
Williams started badly, misfiring her shots into the net and tramlines to drop serve in the fourth and sixth games. She lost the first set in just 28 minutes.
WHIPPING WINNER
Three more forehand errors and a double fault gave Schnyder another break at the start of the second.
From 2-0 down, however, Williams found her rhythm to reel off the next six games level at one set apiece as mistakes crept into Schnyder's game.
In a tight third set, Schnyder created break points in the first and fifth games, but Williams rallied to save them and force a tiebreak.
Williams started the tiebreak badly, mishitting a backhand into the tramlines and then netting a backhand before fighting back to level at 5-5.
A netted forehand, however, gave Schnyder match point, which she converted by whipping a crosscourt winner into the back corner.
Jankovic's win over former French Open finalist Dementieva confirmed her impressive recent form on clay.
The 22-year-old Serb, who has reached at least the quarter-finals at each of her last four claycourt events, dragged the 10th seeded Russian around the court with heavy, angled groundstrokes.
"I used to think that I didn't like clay. I wasn't used to it and I didn't know how to slide," said Jankovic, who triumphed in Charleston last month.
"Then I worked with a Spanish coach, I learned how to slide and I got a lot of confidence, especially after Charleston. I'm starting to like it out there."