Milan moved up to second, six points behind league leaders Napoli, after substitute Giroud crashed in a superb finish from Sandro Tonali's cross in the 89th minute just as it looked like Daniel Maldini had left them trailing even further behind.
In the bedlam that followed his winner veteran striker Giroud, 36, was sent off as he ripped his shirt off in celebration and was given a second booking.
France international Giroud was on a high as he scored twice and set up two more on Wednesday when Milan moved into the Champions League last 16 for the first time in nine years and has now scored nine goals in all competitions.
"It was a great goal but it was a brilliant pass from Sandro," said Giroud to Sky Sport Italia.
"I'm annoyed because I forgot (about the first yellow card), but it is what it is."
On loan at Spezia from Milan, Daniel Maldini curled in a superb leveller just before the hour mark to stun the 70 000 fans present to watch what was expected to be a simple win for the champions, especially after Theo Hernandez had poked the hosts ahead in the 21st minute.
The strike from 21-year-old midfielder Maldini, whose only other senior career goal came against Spezia for Milan last season, would have left Stefano Pioli's side eight points off the pace had Giroud not stepped in with another huge goal.
"He's a great player and he needs to get games. We're waiting for him to come back here as soon as he can," Theo Hernandez said of Maldini to Sky Sport Italia.
A draw with 17th-placed Spezia would have been a big blow for Milan as Napoli went eight points clear earlier on Saturday by beating Atalanta 2-1 in Bergamo earlier on Saturday as they hunt a first Scudetto since 1990.
Milan struck the woodwork twice through Ismael Bennacer and Rafael Leao, while Tonali thought he had given Milan the lead five minutes after Maldini's leveller only to see his superb long-range effort chalked off for a foul by Fikayo Tomori in the build-up.
"It was a hard game, in the first half we had five or six clear chances and when you don't score they can," added Hernandez.