The young Light Blues stormed into an early two-goal lead after only six minutes but their Hoops counterparts recovered to make it 2-2 early in the second half.
However, in the 65th minute, Nathan Young-Coombes headed in what turned out to be the winner.
The match started well for Rangers with Ciaran Dickson curling the ball low into the corner of the net from 20 yards with four minutes on the clock.
When the Hoops failed to deal with a corner into the box two minutes later, Dapo Mebude drove the ball in from inside the box.
Celtic, however, were level within three minutes of the second half.
Rangers goalkeeper Lewis Budinauckas had no chance when striker Cameron Harper guided a shot from 16 yards into the corner within 90 seconds of the restart.
Moments later, Paul Kennedy's free-kick from 25 yards took a wicked deflection off Mebude and sneaked in.
It was Rangers' turn to respond. Celtic keeper Liam Hughes tipped a fine shot from Josh McPake around the post but he was beaten when Young-Coombes headed in from a Kai Kennedy free-kick.
Two minutes later Hughes saved a poor penalty from Kennedy after Brody Paterson had tripped Mebude.
Celtic had the ball in the net in the closing stages but the offside flag was up and Rangers kept ahead until the final whistle.