Substitute Djenepo, a £14million summer signing from Standard Liege, had only been on the pitch for 78 seconds when he cut in from the left to bury the ball into the top-right corner from the edge of the box.
New Seagulls boss Potter had begun his tenure in impressive fashion with four points from a possible six but predicted this would be his team’s toughest fixture to date.
But their task became more difficult with just 30 minutes on the clock.
Seagulls forward Andone miscontrolled the ball near the halfway line and, in his efforts to retrieve it, raked his studs down the leg of Valery.
Referee Kevin Friend was quick to pull out the red card, with replays showing the Romania international could have no complaints.
Albion goalkeeper Mat Ryan then had to be alert to save from Redmond and prevent his side falling immediately behind.
The home supporters, still feeling aggrieved at Andone’s early departure, were left further frustrated seven minutes before the break when VAR disallowed a Brighton goal for the second successive week.
Captain Dunk powered home a header following a right-wing corner, but celebrations were cut short by a review which ruled that fellow defender Dan Burn was in an offside position as he challenged Saints keeper Angus Gunn.
Most inside the stadium, who last week saw Leandro Trossard denied a goal against West Ham in similar circumstances, were left baffled by the decision, with a tweet from the Premier League finally clearing up the confusion.
The away side made their numerical advantage count in fine style 10 minutes after the restart.
Mali winger Djenepo, who had just replaced Valery, collected the ball wide on the left during a Saints counter-attack and advanced to whip home in front of a sold-out away end bathed in sunshine.
Potter, whose only Premier League experience as a player came with Saints under Graeme Souness in 1996, introduced Locadia and Glenn Murray from the bench in the hope of prolonging his unbeaten start.
Dutch forward Locadia had a golden chance to snatch a point with five minutes remaining but he smashed against the outside of the left post from six yards after Murray nodded down a corner.
Saints twice won 1-0 on this ground last season, with their only clean sheet in their last 12 matches coming here in March.
And they dug in resolutely and won by a bigger margin on this occasion thanks to Redmond’s late second.
Winger Boufal, who spent last season on loan at Spanish side Celta Vigo, worked space on the left before squaring across goal for Redmond to tap home and secure the points following successive defeats to Burnley and Liverpool.