The 35-year-old ended a three-year hiatus from the sport by defeating Mark de Mori in January.
Now, at London's O2 Arena, he will face Kosovo-Albanian Gjergjaj, who has a record of 29 victories and no losses.
"I'm so happy to be back in the ring so soon," Haye said in a statement.
"I said I wanted to fight regularly and that's exactly what I'm doing. Every two to three months I want to be knocking somebody out and working my way back into title contention."
Gjergjaj's current winning streak is the second longest in heavyweight boxing behind WBC champion Deontay Wilder.