"Modi had compared Kerala to Somalia, which is infamous for starvation deaths and sea pirates. The pride of Keralites has been hurt," Antony told reporters after exercising his franchise at a booth in Thiruvananthapuram.
"The prime minister had an opportunity to apologise for the remark, but he did not do that," the former defence minister said, adding the remark would affect the BJP's prospects.
"Despite the hectic campaign of the prime minister, the BJP will not be able to open account in Kerala assembly. I am very clear that we will get more seats than we got in last elections," he said,
Modi, while addressing an election rally in Thiruvananthapuram, had compared the infant deaths in some tribal areas in the state with that to Somalia.
However, the BJP defended the prime minister, saying people were "twisting facts" and he had only highlighted the plight of the tribal community in the state and his desire was to improve their living conditions.
Expressing confidence that Kerala will create history by voting to power the United Democratic Front government again, Antony said the Marxist-led Left Democratic Front will have to sit in the opposition for the next five years and BJP will not open its account.
This will be a vote against the politics of violence and communalism and it will be a vote for peace, he said.
The attack on RMP candidate K K Rema at Vatakara will prove dangerous for the LDF, he added.
Sreesanth: 'I will be the best politician in Kerala'
Is the Modi government scared of this man?
'How can anyone not speak out against the genocide of Adivasis?'
'I will not win unless Hindus vote for me'
Meet this unusual BJP candidate from Kerala