Naval patrols had ordered the trawler to stop before it fired on the government boats, a source said, adding that the trawler was sunk in a narrow strip of sea dividing India and Sri Lanka.
"The trawler exploded in a ball of fire ... the navy believed they were trying to smuggle in fuel or explosives," a military official said.
"Three navy sailors escaped with injuries," he added.
Last month, Sri Lankan gunboats and war planes attacked and sank a ship off the northeast coast said to be carrying arms destined for Tamil Tiger rebels.
The rebel Liberation Tigers of Tamil Eelam (LTTE), who are fighting for a separate homeland in Sri Lanka, were smuggling weapons aboard the ship which had no registration marks, the defence ministry said.
Meanwhile, suspected Tamil Tiger rebels Sunday shot dead three men in the frontline town of Vavuniya, 260 kilometres (160 miles) north of Colombo, police said.
They said the victims were members of the majority Sinhalese community who had travelled in