New Zealand Cricket (NZC) also revealed they had received a threat concerning the security of the team should they go ahead with the five-match one-day international tour in Pakistan, due to begin in Faisalabad on November 22.
Batsmen Craig McMillan and Lou Vincent, all-rounder Scott Styris and fast bowler Ian Butler are the four players who will not be making the trip.
The team's video analyst, Zach Hitchcock is also returning to New Zealand.
"The threat is being discussed with the Ministry of Foreign Affairs and Trade (New Zealand) and investigations are proceeding to determine its validity," NZC chief executive Martin Snedden said in a statement.
"We have notified the ICC and the Pakistan Cricket Board of the threat and have entered into a consultation process with them. That process is expected to be completed over the next three or four days."
"We continue to keep the Cricket Players Association and the players fully informed throughout the process."
Earlier this week, media reports suggested that McMillan was baulking at going to Pakistan 18
It was the third occasion in 12 years that New Zealand sides had been close to terrorist action.
The other two were in Sri Lanka in 1989 and 1993.
The decision to proceed with the tour was made by NZC last week following a comprehensive security review by Australian Reg Dickason, an independent security advisor.
NZC determined that New Zealand would proceed to Pakistan at the conclusion of the triangular one-day series also involving India and Australia, but that all players and management would be entitled to make an individual choice on availability for the tour.
"Following the traumatic experience in Karachi when the Black Caps (New Zealand) last toured Pakistan, we agreed that individual team members could make themselves unavailable if they were seriously concerned about safety," Snedden said, adding that those opting out would not suffer any recriminations.
The team to tour Pakistan will not be announced until the validity of the safety threat is determined.