Oscar Trejo's penalty just before half-time was enough to seal victory for Rayo, who will now attempt to make their first-ever final in the competition.
They will be joined in the last four by Valencia, who defeated Cadiz 2-1, Hugo Duro heading in a late winner at Mestalla.
On Thursday, La Liga leaders Real Madrid play Athletic Bilbao at San Mames and Real Sociedad host Real Betis in the other two quarterfinals.
Rayo's winner came in the 44th minute after Alvaro Garcia fooled Franco Russo with a sharp turn in the box and toppled over the sliding Mallorca defender. Trejo guided the penalty confidently into the corner.
Rayo returned to La Liga this season and have enjoyed a superb campaign so far. They beat Barcelona in October and now sit eighth in the table, with survival all but already secured.
They are only the fifth promoted team in the top flight to make the Copa del Rey semifinals this century.
Valencia have won the cup eight times, most recently in 2019 when they overcame Barca in the final, and they scraped past Cadiz thanks to Duro's header 11 minutes from time.
Cadiz had levelled when new signing Lucas Perez scored a penalty to cancel out Goncalo Guedes' opener but Duro ensured Valencia avoided an upset. Cadiz's Juan Cala was sent off late on.