Potter was taking charge of Chelsea for just the second time since replacing the sacked Thomas Tuchel in September.
The former Brighton boss had overseen a 1-1 draw with RB Salzburg in the Champions League and was seconds away from another frustrating result until England midfielder Gallagher rode to the rescue.
Odsonne Edouard had put Palace ahead early on before former Barcelona forward Pierre-Emerick Aubameyang equalised with his first goal for Chelsea late in the first half.
That set the stage for Gallagher to come off the bench and win it for Chelsea deep into stoppage-time, in the process breaking the hearts of the team he played for on loan for the whole of last season.
"It's a very special moment to come on and get the winner. It was written in the stars. Everyone knows how much I loved it at Palace but I'm just buzzing to get my first goal," Gallagher said of his maiden strike for Chelsea.
Relieved to get off the mark after a tricky test in south London, Potter said: "We're delighted with the result. It's a tough place to come. I thought we started quite well but conceded from the first action into the box.
"Credit to the boys, they recovered well and did not let their heads go down. It was nice to get an equaliser and get back in the game.
"There's character, that's for sure. They could've felt sorry for themselves after conceding the goal but the players stood up all the way through.
"There was a collective spirit among the players. We are delighted with the three points."
Edouard met Jordan Ayew's seventh-minute cross and lifted the ball past Chelsea keeper Kepa Arrizabalaga into the top corner.
Raheem Sterling was inches away from a quick equaliser when the Chelsea forward pounced on his own rebound and watched his second effort hit the post.
Palace were furious when Ayew looked through on goal, only to be stopped in his tracks by Thiago Silva, with the Chelsea defender escaping with just a booking.
Silva's presence on the pitch proved vital moments later when Reece James found the Brazilian at the edge of the area and he nodded the pass in the direction of Aubameyang, who spun and fired past Vicente Guaita.
Palace nearly went back in front after halftime when Michael Olise battled his marker before picking out Wilfried Zaha, who could not beat Kepa with a powerful strike.
Gallagher was warmly received by Palace fans when he stepped on the pitch in the 76th minute.
But those friendly sentiments were quickly forgotten when he curled in the decider to spark jubilant celebrations among Potter and his backroom staff.