A first-half penalty from Adri Embarba proved to be the difference as Vallecano moved themselves off the bottom of the standings, although Paco Jemez's team remain six points adrift of safety with three games remaining.
Zinedine Zidane made eight changes to his starting lineup, and it showed as Real barely offered a threat against opponents with the worst defensive record in the division.
Both teams started strongly at Campo de Futbol de Vallecas, but it was Real who created the first shooting opportunity as Marcelo saw a right-footed effort from 25 yards parried away by Alberto Garcia.
Courtois then had to be alert to keep out Pozo on the opposite side of the penalty area, but it was not long before Vallecano opened the scoring from the penalty spot.
Real were still having joy when going on the attack, and their best opening before the break came when Dani Ceballos directed a header wide of the far post after meeting Dani Carvajal's inswinging cross.
The visitors needed to improve after the restart, and there was every sign that was going to be the case when Gareth Bale forced a smart stop out of Garcia after cutting inside onto his left foot.
However, it was Vallecano who continued to make most of the running, and Courtois had to remain alert to deal with a swerving 25-yard strike from former Manchester United man Bebe.
Zidane introduced January signing Brahim Diaz shortly after the hour mark, and the youngster earned the free kick which led to Bale having a set piece tipped over the crossbar by Garcia.
Vallecano would have expected a late onslaught from Real, but it was the home side who looked the most likely to add a second - and decisive - goal during the closing stages of the contest.
Guerra was played in behind the Real backline and the veteran successfully made it around the advancing Courtois, but Vallejo got back to make the clearance on the line.
Six minutes were added on at the end of the game, but Vallecano never found themselves in any real danger of letting go of a result which could provide them with the springboard to launch an unlikely survival bid in the coming weeks.
RAYO VALLECANO (4-2-3-1): Garcia; Tito, Ba, Galvez, Moreno; Agbo, Suarez (Medran 77'); Embarba, Pozo (Trejo 85'), Bebe (A.Garcia 78'); Guerra
REAL MADRID (4-4-2): Courtois; Carvajal, Varane, Vallejo, Marcelo; Kroos, Modric (Isco 74'), Llorente, Ceballos (Diaz 61'); Bale, Mariano (Vazquez 82')