The Iron Man star said that just because he was speaking about a film meant that he had to be grilled about his own past, Contactmusic.com reported.
The 50-year-old actor said that what he has to do in the future is to give himself permission to say, "That is more than likely a syphilitic parasite, and I need to distance myself from this clown.
"Otherwise, I'm probably going to put hands on somebody, and then there's a real story".
The Avengers: Age of Ultron star stormed out from an interview with Channel 4's Krishnan Guru-Murthy last week over personal questions.
Photograph: Paul Drinkwater/Getty Images
ALSO READ: Robert Downey Jr didn't need to act like a jerk
This is India! Dalit school students forced to clean toilets
Exclusive! Saving the Apu Trilogy
Sunil Narine banned from bowling off-spinners
Don't mess with me!
Should the Railways take a leaf out of Lalu's book?