Anders Frisk says worries over his own and his family's safety especially since the contentious Champions League Barcelona v Chelsea game prompted him to quit refereeing.
"The threats toward me have escalated the last few weeks after the Barcelona-Chelsea game," Frisk said on Sweden's football federation website on Saturday.
Frisk said he had received personal death threats since the match.
"It's not worth carrying on... My safety and the safety of my family goes before anything else. These last few weeks have been the worst of my life."
The first-leg knockout clash had ended in controversy after Chelsea accused Barcelona manager Frank Rijkaard of trying to influence the match by speaking to Frisk at halftime.
Chelsea, who were leading 1-0 at halftime, were doubly angered when Frisk harshly sent off striker Didier Drogba early in the second half. Manager Jose Mourinho refused to attend the mandatory news conference and his conduct is being investigated by UEFA.
"I still say that what I did in Barcelona was correct,"
Frisk told the daily Aftonbladet.
The London club went on to win the second leg 4-2 to claim a 5-4 aggregate victory and a place in the quarter-finals.
"Chelsea have supporters in many parts of the world, so all of them (the threats) didn't come from England," Frisk said.
"Soon it will be impossible to referee a Champions League match if the clubs can't accept defeat or a player being sent off," said Frisk, who has been involved in controversy before.
Last September at Rome's Rome's Olympic Stadium Frisk was struck on the head by an object thrown from the crowd during a Champions League match between AS Roma and Dynamo Kiev. He abandoned the game saying the host club could not guarantee the safety of the officials.
In December Claudio Ranieri, at the time Valencia's coach, openly criticised the Swede after his side went out of the Champions League following a 2-0 home defeat by Werder Bremen.
The Italian said that after watching the game he understood why Frisk had been hit on the head in Rome.
Frisk is not the first referee to receive serious threats.
Swiss referee Urs Meier was the target of British fans after disallowing England's 90th minute goal in the Euro 2004 quarter-final against Portugal.
"The same kind of threats have hit me. Urs kept on going but I can't take it, I simply don't have the strength for it," said Frisk, who could have continued refereeing for three more years.