The victory gave them an unassailable 3-0 lead in the five-match series.
Hendricks, 28, who has played 12 Twenty20 internationals since 2014, became only the third South African batsman after Colin Ingram in 2010 and Temba Bavuma in 2016 to reach 100 runs in a first ODI innings.
Jean-Paul Duminy's 92 off 70 balls as well as half-centuries by Hashim Amla and David Miller, also contributed to South Africa's imposing total after being put into bat first.
Amla gave South Africa a brisk start with 59 in as many balls before being bowled by Thisara Perera.
Hendricks then built crucial partnerships including a 78-run fourth-wicket stand with Duminy whose swashbuckling innings included eight ‘fours’ and four ‘sixes’.
Duminy was later abetted by fellow left-handed batsman Miller. The pair added 103 runs for the fifth wicket.
Miller's 51 and Andile Phehlukwayo's unbeaten 24 off 11 balls gave South Africa 98 runs in the last seven overs.
In reply, Sri Lanka were never allowed to settle and threaten. South Africa’s bowlers took wickets at regular intervals. Lungi Ngidi returned the best figures of four for 57 while fellow paceman Andile Phehlukwayo claimed three for 74.
Sri Lanka will attempt to salvage some pride from the series in the fourth match on 8 August in Pallekele.