Opener Dean Elgar, who made 103 in the first innings, was the first batsman to fall as he tried to go for an ambitious drive off Herath but nicked it behind to wicketkeeper Dinesh Chandimal.
Alviro Petersen (32) was given caught behind off Perera but the batsman asked for a review. TV replays remained inconclusive and in the end the decision of on-field umpire was upheld.
The batsman walked off the field, shaking his head in disappointment.
South Africa had wrapped up the Sri Lankan innings in the fifth over of the day with paceman Morne Morkel (3-49) sending back overnight batsman Herath for 19.
The second and final Test will be played in Colombo from July 24-28. abh/ac The prolific left-hander also shared 96 runs for the unbroken second wicket with Kaushal Silva (37) to take Sri Lanka to 110-1 at stumps.
The hosts need 260 runs with nine wickets in hand to pull off a magical come-