Mainz substitute Karim Onisiwo had cancelled out Suat Serdar's opener.
Amine Harit struck a fine 89th-minute winner as Schalke beat Mainz 2-1 at the Veltins-Arena to move up to second place in the Bundesliga.
Harit provided the assist for the opening goal in the 36th minute when his pass found Suat Serdar in the box and the ex-Mainz player lashed the ball past Robin Zentner.
The visitors drew level with 15 minutes left through substitute Karim Onisiwo, who collected the ball from Jean-Paul Boetius, turned and sent a curling shot into the net.
Harit then secured all three points for David Wagner's side with a delightful finish, the Morocco international dispatching the ball into the corner of the net from the edge of the area with the outside of his boot.
Schalke go level on points with leaders RB Leipzig ahead of the weekend's action, while Mainz remain 16th.