The Court of Arbitration for Sport (CAS) decided to uphold the ban imposed by the U.S. Anti-Doping Agency after Edwards tested positive for the stimulant nikethamide in Martinique last April.
CAS said in a statement: "As a consequence, Ms Torri Edwards remains suspended until July 17 2006."
After the positive test Edwards, who had been due to run both the 100 and 200 in Athens, said she had been given glucose tablets by her physical therapist who did not know they contained a banned substance.
But the CAS statement said it would have been clear to any person reviewing the tablets that they contained more than one ingredient, even though they were labelled in French.
"According to the panel, there is an obligation and a duty for all athletes and more particularly to top-level athletes to ensure that no prohibited substance enters his/her body, tissues or fluids.
"The panel found that Torri Edwards was negligent in failing to enquire whether the product contained a prohibited substance."
International Association of Athletics Federations (IAAF) spokesman Nick Davies welcomed the decision.
He said: "The IAAF is pleased that CAS has upheld the decision made by its doping panel which is consistent with the federation's rules.
ALSO NEGLIGENT
"CAS also found that the athletes and their support staff had been negligent.
"The credibility of our sport depends on athletes taking full responsibility for any substance they take."
The CAS panel had met for six hours on Monday to consider an appeal by the 27-year-old American against the ban.
The athletics programme begins at the Olympic stadium on Friday after the men's and women's shot put finals at Ancient Olympia two days earlier.
Edwards was confirmed as world champion when her fellow American Kelli White accepted a two-year ban after admitting taking several performance-enhancing drugs.
However, Edwards in turn was banned for two years by the U.S. Anti-Doping Agency (USADA) last Wednesday.
A U.S. arbitration panel had concluded there could have been exceptional circumstances and referred the case to the IAAF.
The IAAF disagreed and its recommendation of a two-year ban was accepted by USADA.