Empire, on the other hand, doesn't rely on any future films for its own plot to make sense. It assumes that you've watched the previous entry in the series, but that's all. It doesn't bring up any plot points for RotJ to explain. Sure, Vader may have revealed himself to be Luke's father, and Han may be frozen in carbonite, but the heroes are safe and are ready to go rescue Han.
Except Han is one of the heroes (the most popular of the protagonists, in fact) and is very much not safe. Empire was roundly criticized at the time of release for being too dark, too reliant on its cliffhanger ending, and assuming too much that its audience would go see a third movie to finish the story. Empire is a singular movie in comparison to the rest of the SW movies, but it is very much not standalone in any sense. This is one of its strengths to those who grew up with it, but it was not considered such in 1980, much like Rogue One is criticized now for expecting you to have seen one of the most widely watched motion pictures ever made.
It was the part that felt the most flagrant and masturbatory for me it saddens me that you're 100% correct
I was far more invested in the characters than Mike and Jay were, even thinking the last moments on scarif were poingnant, and then that fuckin red lightsaber just had to ignite at the same moment as the erections of all the Kylo Rens in the audience...
Oh give me a break. That whole sequence is done like a horror movie. You're not supposed to be rooting or cheering for Vader in that sequence. I don't care if a bunch weirdos do, either. It's an effective sequence and it only adds to Vader's malevolence in the subsequent films.