Until last season, Manchester United had never lost in the Premier League at Upton Park but the Londoners, who also triumphed at Old Trafford in May, made it two in two seasons with late goals by defenders Anton Ferdinand and Matthew Upson.
United still lead the way on 45 points but Arsenal, on 44, can go above them if they win at Everton in Saturday's late game. Chelsea are third on 41.
Liverpool, fourth on 36 and fifth-placed Manchester City (35) meet at Eastlands on Sunday.
The game of the day was at White Hart Lane where Bulgarian striker Dimitar Berbatov scored four as Tottenham Hotspur beat Reading 6-4 with seven goals coming in a crazy 20-minute second-half spell.
It was the second time this season that Reading had scored four away from home and lost following their 7-4 defeat at Portsmouth in September.
Across London, Manchester United took the lead when Ryan Giggs crossed for Cristiano Ronaldo to head his 18th of the season after 14 minutes.
PENALTY MISS
The Portuguese winger should have settled it from the spot in the 66th after a Jonathan Spector handball but shot wide.
West Ham, who had always been in the game, equalised with a header by Ferdinand, brother of United's Rio, after a 77th minute corner and won it eight minutes from the end with Upson's header from a free kick.
"The players aren't perfect - we had a perfect performance in the last game but this was a disappointment," United manager Alex Ferguson told Sky Sports.
"I think our team struggled today, West Ham were very competitive and aggressive, we can't complain, I think they were the better team."
Chelsea, without a host of first-choice players through injury and suspension, also struggled for long spells against Newcastle despite Michael Essien putting them ahead after 29 minutes.
Nicky Butt scrambled a 56th-minute equaliser but Newcastle were left down and out after Kalou, who seemed clearly offside, got the winner three minutes from the end.
"It was a clear, clear offside by two or three yards, you don't know why the assistant referee cannot see that player in that position and not put his flag up," Newcastle manager Sam Allardyce told Sky Sports.
"For us it's a severe blow after a performance that was worth at least a point."
Chelsea manager Avram Grant pointed to a victory that came almost exactly a year after the west London club's title chances were hit hard by a run of three draws.
"I think this was one of the most important victories, when you remember what happened at this time last year," he said.
There was little hint of the chaos to come as Reading led 2-1 at Tottenham after an hour.
Then the floodgates opened and at the end of it all Berbatov's four and one each for Steed Malbranque and Jermain Defoe settled an extraordinary encounter.
"It is difficult to explain," Spurs assistant manager Gus Poyet told Sky Sports.
"Yes, we did a few things that are not very good but others were good - like the six goals."