But after challenging early on the Canadians faded and finished fifth.
Earlier, there was double gold for Romania, triumphant in the women's eights and the lightweight women's doubles and Denmark took their first gold of the Games.
But the highlight was the men's eights where the American crew punched the air in delight as they crossed the line and screamed for joy as their cox threw himself into the arms of his exhausted stroke man.
"They are the best guys we have," coach Mike Teti said. "We took a big risk because we had a good squad (in the fours) at Lucerne (at the World Cup) but it's 40 years since we won this (the eights) and we just had to get that monkey off our backs."
The U.S. four beat Britain, who won the Olympic title on Saturday, in Lucerne.
"When they came back from Lucerne I thought they were going to want to do the four but they said they want to do the eight, they wanted to end this. That's what makes it so special."
The U.S. held off the Netherlands and Australia.
GOOD DAY
In a good day for the Romanian team, their women's eight won their third consecutive Olympic eights title to give Elisabetha Lipa her fifth Olympic gold. They beat the U.S. favourites by half a length.
Earlier, Constanta Burcica won her third Olympic gold medal as Romania retained the lightweight women's double title.
Burcica and Angela Alupei, who won her second gold, came from behind to take the lead after 1,500 metres.
In the men's event, Poland's Robert Sycz and Tomasz Kucharski won gold by clear water to retain the title which they took in Sydney and Greece came from behind to take bronze to raucous applause.
The Poles' season was marred by injury and Sycz said coming into the Games they had hoped for a place on the podium.
"However after our heats the target was the Olympic gold medal," he said.
GERMAN DOMINATION
Germany maintained their domination of the women's quadruple sculls event, beating favourites Britain to retain their Olympic title.
The Germans have been Olympic champions since 1988 when the event was added to the Games programme and their coaches reorganised their crews to strengthen the quad after they lost to Britain in a pre-Olympic world cup event.
In the men's quad sculls, Russia stunned the favourites Germany to beat the Czech Republic and Ukraine. Germany, three times world champion. finished fifth.
In the men's lightweight coxless four, double world champions Denmark won a thrilling race to take Denmark's first Olympic gold of the Games. The Danes had just under a length lead by the finish, holding off Australia.
The win gave Denmark's Eskild Ebbesen his second Olympic gold and Thor Kristensen, Thomas Ebert and Stephan Moelvig their first.
"The Danes have dominated this event for the last 11 years, they've won eight of the last 11 world titles and we are just unlucky to have been in an era a bit like when Ed Moses was running in the 400 metre hurdles," Australia's Simon Burgess said.