They said the militants fired pistol shots from a close range at Ayub Wani, a truck driver while he was leaving a mosque after offering prayers in Kakpora, 25 km from Srinagar.
"Ayub died on the spot", a senior police officer said. He said two brothers of the slain driver are working with state police.
No group has yet claimed responsibility for the killing.
Militants open fire at NC activist in Kashmir
Heavy snow cuts off Kashmir Valley
Kashmir: Two children killed by explosive device
Curfew lifted from parts of Kashmir Valley
Three Hizb terrorists killed in JK gunfight