Nikola Katic's second-half header gave Rangers their first win at Celtic Park in almost a decade and massively boosted their Ladbrokes Premiership title hopes.
In a thrilling first half, Gers keeper Allan McGregor saved a Ryan Christie penalty in the 32nd minute before Ryan Kent fired the visitors ahead four minutes later.
The home side levelled in the 41st minute when Callum McGregor's shot appeared to come off Odsonne Edouard's arm and nestled in the net.
However, Rangers were not to be denied and in the 56th minute defender Katic, who had given away the penalty by grabbing a handful of Christopher Jullien's shirt, powered in a header from a Borna Barisic corner for a 2-1 win.
Gers' Colombian striker Alfredo Morelos was sent off by referee Kevin Clancy in added time for picking up a second yellow card, that one for simulation, but with their first success in the east end of Glasgow since October 2010, they moved to with within two points of the champions going into the winter break, but with a game in hand.
A minute's applause in tribute to former Celtic captain Duncan MacKay who died last week, preceded another contentious and incident-packed Old Firm encounter
Rangers, with Steven Davis and Morelos back for Scott Arfield and Jermain Defoe, started well, taking the game to a nervous-looking home side, albeit the final pass was missing.
Morelos got a couple of shots away from distance but keeper Fraser Forster gathered both with ease before he kicked away a shot on the turn from Davis after the Govan men forced a couple of corners.
There was a huge cheer from the home fans when referee Clancy pointed to the spot and indicated that Katic had been pulling the shirt of Jullien at a corner.
Christie blasted his shot to Allan McGregor's left but the veteran keeper dived to parry the ball away before it was cleared, although moments later Davis cleared a Jullien header off the line.
⛔️ @therealgreegsy1 🙌 pic.twitter.com/mc1EZC3oen — Rangers Football Club (@RangersFC) December 29, 2019
There was further reason for the Hoops disgruntlement when, following another measured move up the park, Kent swept a Barisic cross in off the post from 14 yards.
Celtic fought back but Rangers were furious when Callum McGregor's drive after picking up a loose ball seemed to hit Edouard's arm to wrong-foot Allan McGregor in the Gers goal.
Steven Gerrard's men took a sense of injustice into the second half and in the 49th minute Morelos scooped a cross from Kent over the bar from six yards, failing again to break his scoring duck in the fixture.
The Light Blues remained in control and deservedly went ahead again.
Katic had failed to get his head on a Barisic corner from the right but when his fellow Croatian delivered from the left seconds later, he got away from Kristoffer Ajer to header past Forster.
In the 65th minute Davis again was in the right place at the far post to head clear Jullien's header from a Christie corner, before Allan McGregor pushed away a goal-bound drive from Parkhead left-back Boli Bolingoli.
Ntcham and Nir Bitton replaced Johnston and James Forrest and with 10 minutes remaining, and Celtic still toiling, Arfield replaced Glen Kamara in the Gers midfield and Leigh Griffiths came on for Christie.
Celtic kept pushing but Rangers put all hands to the pump to take the points to make a tense title race even tighter albeit they ended the game with 10 men after Morelos, booked earlier for a foul on Scott Brown, was shown a second yellow after collapsing in the Hoops box.
