The Udappuwa area is ethnically Tamil, but the end of fighting between the government and Tamil rebels in 2009 after decades of warfare has failed to bring any peace dividend.
"Some of those who are leaving for Australia are actually trying to escape loan sharks," Quintus Fernando, 56, a local community leader, told AFP.
"They charge five times more than commercial banks," he said, explaining that most fishermen are forced to borrow money to buy their equipment because they do not have any collateral to take out loans from banks.
Fishing restrictions, a legacy of the decades-long separatist war, have also persuaded locals that the industry will never prosper and that they should try to migrate.
"It is three years after the end of the war, but fishermen are still required to get a 'pass' from the navy before setting out to sea.
We can't just row out to fish," Fernando said.
Naval authorities insist that the checks are necessary to prevent the illegal boat traffic abroad.
One