United's attacking force was toothless for 93 minutes at Old Trafford but found the breakthrough at the death as all three of Rangnick's substitutes combined when Anthony Martial and Edinson Cavani teed up Rashford for a tap in at the back post.
"The atmosphere (in the dressing room) is amazing. Quite rightly the boys were celebrating, they know what a massive win this was," said Rangnick. "Those are the best kind of wins when the other team has no time to come back."
The victory lifts United a point above the Hammers and sees the Red Devils leapfrog Tottenham and Arsenal in the battle for Champions League football next season.
The winner was also a huge moment for Rashford, who has looked devoid of confidence in recent months.
The England international scored for the first time since October in a 3-1 win at Brentford on Wednesday and was in the right place at the right time for what could be a decisive moment in United's desperation not to miss out on the riches and prestige of Champions League football next season.
Rashford was again left out of Rangnick's starting line-up in favour of 19-year-old Anthony Elanga.
However, he was the first of three changes made by the German after the break, who all played a part in the winning goal.