Delaney was penalised for a high foot despite Phil Jagielka handling the ball in the same challenge, and Romelu Lukaku fired the resulting free kick past Steve Mandanda to give the hosts the lead at Goodison Park.
It was one of two notable controversies in the match as later Delaney had a goal ruled out for offside, although the stand-in Palace skipper revealed that the flag was correctly raised against James McCarthy.
"I think [Phil Jagielka] put his head down a bit [for the free kick for Lukaku's goal]. A bit harsh. But we should have jumped in the wall and didn't," he told Sky Sports News.
"We spoke about whether to jump having seen [Manchester City's] Kevin De Bruyne roll the ball under the wall last week. It was my call. But we have great camaraderie here and came back well.
"The referee said McArthur was offside and interfering with the goalkeeper [for the goal ruled out for offside]."
Palace are now unbeaten in their last five Premier League matches.