Australia had been put into bat at the Wanderers, but were soon a man down.
David Warner – returning to South Africa for the first time since serving a ban for his involvement in the ball-tampering scandal during a Test match in 2018 – hit his first ball for four but top-edged Dale Steyn and was caught at long leg for four from his second delivery.
South Africa captain Quinton De Kock was bowled by Mitchell Starc in the third ball of the reply and the hosts then slipped to 40 for four in the seventh over, before Agar produced his own magical moment.
The left-armer had former Proteas skipper Faf du Plessis (24) caught on the boundary, then trapped Andile Phehlukwayo lbw and forced Dale Steyn to edge to slip as he became only the second Australian to claim a T20 hat-trick after former paceman Brett Lee.
Kagiso Rabada offered some late resistance with 22 off just 19 deliveries, which included two sixes.
South Africa were all out for just 89 off 14.3 overs – their lowest score in a T20 international.
Agar – who almost took another treble across two overs later in the innings – finished with five for 24, while Pat Cummins and leg-spinner Adam Zampa also both picked up two wickets.
The second T20 international of the three-match series is in Port Elizabeth on Sunday.