The French forward exchanged passes with Memphis Depay before finishing clinically to earn Atletico the three points.
The victory helped Atletico cut the gap on third place Real Sociedad to two points after the Basque side drew on Saturday.
Atletico paid tribute to Athletic's 125th anniversary by wearing their away shirts, with Atletico having been formed as a branch of Athletic in 1903.
Elsewhere, Rayo Vallecano and visitors Sevilla shared a 1-1 draw, with the hosts, sixth in the table, unable to exert maximum pressure on the sides above them in the quest to secure European football next season.
Sergio Camello, on loan at Rayo from Atletico Madrid, missed a golden opportunity to level for the hosts, while both Sevilla's Youssef En-Nesyri and Rayo's Raul de Tomas had goals ruled out for offside late on.
Meanwhile, Sergi Darder's late strike secured Espanyol a 1-0 win at bottom-of-the-table Elche.