The Blues escapes a potential FA Cup upset by the skin of their teeth as they require extra time for a 2-1 win over Plymouth.
Gillesphey stunned Stamford Bridge with his opener, which Azpilicueta cancelled out in the first half.
The Blues then had to wait until the 106th minute for an eventual winner through Alonso, with Hardie saw a penalty saved with two remainings remaining in this fourth-round tie.
CHELSEA (3-5-2): Kepa Arrizabalaga;
Cesar Azpilicueta, Antonio Rudiger, Andreas Christensen;
Malang Sarr, Jorginho, Mason Mount, Mateo Kovacic, Hakim Ziyech;
Romelu Lukaku, Callum Hudson-Odoi.
PLYMOUTH ARGYLE (3-5-2): Michael Cooper;
Macaulay Gillesphey, James Wilson, Dan Scarr;
Joe Edwards, Panutche Camara, Jordan Houghton, Danny Mayor, Conor Grant;
Jordan Garrick, Luke Jephcott.
Goals and Key moments:
0-1. The visitors are in dreamland as Gillesphey nods into the bottom-right corner. Schumacher’s men win a free-kick on the left flank, which Houghton delivers towards the six-yard area, and the centre-back does the rest as he flicks past Arrizabalaga.
1-1. Kovacic is patient as he works towards the right flank, where Mount drills a low cross towards the near post. Azpilicueta finds himself unusually far forward and finishes past Cooper from close range with an audacious, flicked finish.
2-1. Havertz exchanges a one-two on the left flank with Werner before making it to the byline and pulling back towards the penalty spot. Alonso ghosts into the area and slides a right-footed strike into the bottom-right corner past the unsighted Cooper.
PK SAVE. After Sarr makes a mistake in giving away a penalty, Hardie hovers over the chance from 12 yards out. He slides towards the bottom-right corner, but Arrizabalaga guesses the right way and saves.
So, where else do Chelsea and Plymouth go from here?
Chelsea now heads to the Club World Cup, where they will play in the semi-final on Wednesday. Plymouth, meanwhile, travels to Crewe the day before as they return to their League One campaign.
Likewise, for more football news like this, click here.