Hoops midfielder Callum McGregor thundered in the opener from the edge of the box in the 16th minute before Livi captain Jon Guthrie levelled eight minutes later after a Fraser Forster fumble.
Midfielder Scott Robinson, who had scored in the Lions' 2-0 win over the Hoops in October, notched less than a minute after the break. However, at the death, substitute Rogic slammed in an equaliser to take Celtic 13 points clear of Rangers, who lost 1-0 at home to Hamilton.
Rangers have a game in hand but Celtic showed just how determined they are to make it nine titles in a row this season.
All eyes were on West Lothian as Livi attempted to repeat their previous shock win over the champions, who had recorded their 34th consecutive domestic cup-tie victory on Sunday with a 1-0 William Hill Scottish Cup quarter-final win at St Johnstone.
Attacker Elyounoussi returned to the Hoops side to partner striker Odsonne Edouard at the expense of Leigh Griffiths, while Robinson replaced Jack McMillan and former Celtic defender Efe Ambrose made his home debut.
The visitors took the lead moments after Hoops keeper Forster had tipped a long-distance effort from Robinson past a post.
The Parkhead side survived the corner and when they raced up the other end, Livi defender Steve Lawson failed to connect with his clearance from Greg Taylor's cut-back and the ball landed at the feet of McGregor, who quickly switched the ball on to his right before driving high past Livi goalkeeper Robby McCrorie, on loan from Rangers.
Forster was at fault for Livingston's unexpected leveller.
The Southampton loanee failed to gather left-back Ciaron Brown's long throw-in under pressure from Guthrie and the Livi defender simply hooked the ball over his head and into the net.
There were some Celtic looks towards referee Willie Collum to blow for an infringement but the goal stood.
The champions then had two good chances to restore their lead.
In the 35th minute Celtic defender Christopher Jullien headed a Ryan Christie free-kick over the bar. Edouard then crashed a shot off a post after being set up by McGregor, who had robbed Lawson on the touchline, before McCrorie punched away a shot from Taylor with the last action of the first half.
Celtic and their fans were stunned within a minute of the second half when striker Lyndon Dykes outmuscled Jullien on the right-hand side and rolled the ball back for Robinson to drive into the net from 14 yards.
Forster saved Aaron Taylor-Sinclair's drive from wide with his feet moments later as Celtic wobbled before storming back.
Dykes missed the target with a curling shot after Steve Lawless played him in after picking up a loose Jullien clearance.
But Celtic kept piling forward and only a timely tackle by Guthrie thwarted Edouard as he set himself to shoot just 12 yards from goal.
It was backs-to-the-wall defending by the home side, but in dramatic fashion Rogic converted an Edouard cut-back in stoppage time as Celtic were again rewarded for their never-say-die spirit.