A police official said the prisoners fought over the volume of a TV set and hit Shakrullah, 50, with a large stone.
The police said 'three or four' prisoners were responsible for killing the life convict, who is from Sialkot in Pakistan's Punjab, and have registered a case of murder.
"A Pakistani prisoner was allegedly murdered today at the Central Jail. The matter will be investigated by a judicial magistrate and also by the police," Rajasthan Director General of Police Kapil Garg told PTI.
The incident comes amid heightened tension between India and Pakistan after the Pulwama terror attack in which a car bomb killed 41 Central Reserve Police Force personnel.
Inspector General (Prisons) Rupinder Singh said Shakrullah had been lodged in the jail's special cell since 2011.
Earlier, he and seven others were held in a Punjab jail, accused of radicalising young people.
The module operated under the directions of its handlers in Pakistan, Jaipur Police Commissioner Anand Srivastava said.
Rajasthan's anti-terrorist squad had brought the eight accused from Punjab for interrogation in similar charges and booked them under Unlawful Activities (Prevention) Act, the official added.
In 2017, the LeT operative was sentenced to imprisonment for life.
Srivastava said the fight on Wednesday was over reducing the volume of a TV set in the jail.
Senior administration and police officials rushed to the jail after learning about the death.
Rupinder Singh said the post mortem will be conducted on the jail premises under judicial supervision.
Meanwhile, Pakistan expressed concern and sought response from India on reports of the killing.
According to Pakistan's Foreign Office (FO), Shakrullah was beaten to death by a group of Indian inmates in retaliation of the Pulwama incident.
"Pakistan is gravely concerned at media reports regarding the brutal killing of a Pakistani prisoner," the FO said.
It said the Pakistan High Commission in New Delhi has officially raised the issue with the Indian authorities and requested them to urgently authenticate the report.
The response is still awaited, it added.
Pakistan also called upon the Indian government to ensure foolproof security to all Pakistani prisoners lodged in Indian jails as well as Pakistani visitors to India.
'US does not want India, Pakistan to go to war'
Pakistan needs to take firm actions against Jaish
'The war has come home to us'
'Pak has to be given a message that we will keep hitting you'
'The Kashmiri has had enough of Pakistan'