During the match, Conor Gallagher's shot was blocked by the arm of West Ham's Tomas Soucek, but the referee did not award a penalty. The game ended in a 1-1 draw.
Following the incident, former Chelsea player Joe Cole and BT Sport pundit suggested that Potter should have caused "a bit of uproar," while Rio Ferdinand compared him to Jose Mourinho.
However, Potter, whose team is set to play the first leg of their Champions League last 16 tie at Borussia Dortmund later on Wednesday, said that he had a responsibility not to contribute to the abuse of referees. He also highlighted the connection between media criticisms and the problems with referees at the grassroots level.
"The same media are talking about me being angrier but then running stories about problems with referees at grassroots level," Potter said. "They don't see the connection."
Potter acknowledged that it is natural for people to lose their temper in such an emotional situation, but he emphasized that he has a responsibility to himself, to Chelsea, and to the game to act in a way that is the right thing for him.
The BBC conducted a questionnaire that received over 900 responses from amateur referees in England. The survey revealed alarming levels of abuse and intimidation, including numerous cases of death threats being made.