But that hope was ended in the most spectacular style as Celina found the top corner from 25 yards, and Barrie McKay added another six minutes from time to see Swansea through to the last 16 of the FA Cup in consecutive seasons for the first time since the mid-1960s.
This was a special occasion for Gillingham's Swansea-born manager Steve Lovell, who played for his hometown club in the 1980s.
Lovell's father was also a Swansea season-ticket holder, but the former Wales forward was denied a happy homecoming as Swansea showed just why there are 34 places separating the two clubs in the league ladder.
There was a poignant moment before kick-off as Swansea fans paid tribute to Emiliano Sala, who was signed by their great rivals Cardiff just days before going missing in a plane travelling over the English channel.
Gillingham had upset Cardiff in the previous round but Swansea, and McBurnie in particular, were intent on making a fast start.
McBurnie struck the post with a curled effort from 20 yards after six minutes and then diverted Matt Grimes' free-kick inches wide as the League One visitors caught their breath.
But McBurnie was not to be deterred and, running on to Leroy Fer's cross, he planted a powerful header past the despairing dive of Gills goalkeeper Tomas Holy.
Gillingham managed a low Tom Eaves effort after Fer lost possession near halfway, but Swansea threatened with almost every attack.
McBurnie doubled the advantage after 32 minutes – his 13th goal of the season – when McKay flicked Connor Roberts' pass into his path and he finished unerringly from 15 yards.
The Scotland striker came close to his first Swansea hat-trick straight after the break when he headed Fer's cross on to the roof of the net.
But Gillingham were boosted in the 51st-minute when Elliott List kept possession well near the corner flag and Rees met Mark Byrne's cross to score with a flashing header.
Gillingham were suddenly asking questions with Eaves a useful focal point to their attacks, but McBurnie saw penalty claims turned down and Holy was alert to handle Grimes' low drive.
Celina's stunning drive past a helpless Holy gave Swansea breathing space before McKay put the gloss on victory when he tapped home substitute Dan James' cross at the near post.