Just one spot remains available in the top six heading into the final matchday, with Preston North End the only team realistically able to overhaul Derby County due to the Rams' superior goal difference compared to the other members of the chasing pack.
Millwall, among those now expected to miss out on a playoff spot following this defeat, were behind inside 12 minutes when Assombalonga - preferred to Patrick Bamford up top - found space at the second attempt and struck the ball past Jordan Archer at his near post.
Daniel Ayala remained alert to get to the ball ahead of Jake Cooper at the back post, with Millwall - now without a clean sheet in 11 matches - reminding their opponents that they were still in the contest.
A couple more half-chances went begging for the hosts prior to their killer second, including one for Adama Traore, but the Lions never looked like punishing Boro.
The second goal came via Howson, who walked the ball over the line after visiting keeper Archer made a meal of Downing's corner.
There was to be no route back into the game for Millwall, as Lee Gregory failed to make any true contact with his headed chance, before getting a weak shot away in the closing stages.
Boro felt that they should have had a chance to add a third to their tally, only for referee Paul Tierney to wave away Besic's penalty appeals.
MIDDLESBROUGH (4-3-3): Randolph; Shotton, Ayala, Gibson, Friend; Clayton, Besic, Howson; Traore, Assombalonga (Bamford 77'), Downing (Fabio 81')
MILLWALL (4-4-2): Archer; Romeo (McLaughlin 38'), Cooper, Hutchinson, Meredith (Ferguson 83'); Wallace, Williams, Tunnicliffe, Marshall (Elliott 54'); Morison, Gregory