The Midlands side are rooted to the bottom of the Premier League table having amassed just three wins all season, with relegation to the Championship all but certain.
"I've said since Christmas that we're going down and for me it's so unbelievably sad, so difficult to put into words," 92-year-old Ellis told The Telegraph.
"The famous Aston Villa, 1874 and all that, with facilities as good as anywhere in the country, 40,000 odd people in comfort, going into the old Second Division.
"Villa will be the biggest club to ever be relegated, in my mind. It's been my life for such a long time, nearly 32 years of them as chairman, and it's breaking my heart to see. I won't shed a tear on Saturday because I did that at Christmas. We've just got to accept it and get on with it."
Villa's fate could be confirmed this afternoon if they fail to take anything from their game with Bournemouth in Birmingham.