One of the 100-odd lobster breeders in the area, Liew Si Mun, said the men approached fish farms to ask the workers if their "towkays" were around.
Chan from Perak was abducted outside his fish farm in Kg Sapang in Kunak, after he was awakened by the barking of his dogs at 12.40am.
According to Liew, Chan's fish farm is out in the open sea, about 500m from the mangrove forest inland, where his (Liew's) farm is located.
"Dorang tanya-tanya dulu itu budak pekerja baru dorang culik (They asked the workers questions before taking Chan)," he told Daily Express, Tuesday.
He said the situation would be different if the farm was built closer to the mangrove, which he felt much safer than exposed in the sea and closer to land.
"Tidak takut la. Biasa saja. (No I'm not afraid. I'm not swayed)," Liew said.
Liew who is the proprietor of Sapang Marine, at present employs about nine local workers of Bajau and Suluk descent.
Meanwhile Chan's working partner at Darvel Bay Hybrid Aquaculture Sdn Bhd, Khoo Mun Huah has moved to beef up security on the fish farm.
"We cannot stop operating," he sighed.
Khoo said their operations cannot be seized because they hold 80 per cent share of the country's supply chain of aquaculture fish, besides being one of the anchor companies in the State.
He said for 20 years there had never been any kidnapping.
He said security in the area was lax, but the blame should not go to Eastern Sabah Security Command because the coastal areas are massive.
He also noted of the Esscom Director-General Datuk Mohd Mentek visit to the farm and was told that they will conduct a study on how to deal with the present scenario and measures to curb future kidnappings.
"I also feel that the media should not highlight the figure of ransoms paid to the kidnappers, it will only drive more kidnappers into Sabah," he said.
This is the third such case Sabah's East Coast where police believed as another kidnap-for-ransom case.
On April 2, 29-year-old Chinese national student and a resort worker was kidnapped and later released on May 30 and another abduction of a 34-year-old Chinese national manager of a cage fish breeding firm in Pulau Batik Lahad Datu.
Based on the Esscom Intelligence sharing network with the Philippines authorities, there are about 14 kidnap-for-ransom group currently still active in the Southern Philippines.