Metroid fan here. I'll take a crack at this question as it is a good one.
As cheesy as it may sound, those games are Metroidvania, Metroid is an action adventure.
That's a big difference in my view as those game are taking what I love about Metroid and pushing them to the max.
Like Demon's Souls, Metroid, because weve made it into its own genre, isn't trying to reinvent itself. Thus, I don't think Metroid is trying to be the best Metroidvania.
Games like Guacamole and Hollow Knight are best example of the genre.
-----
Now as for Metroid on it own. It's a game that has so much potential in world building, multiplayer, deeper story, yet, it never steer to far from it isolated, mysterious, deep dungeon crawling, element. So when something like dread or a direct sequel comes out, we take it.
We've accepted Metroid as a franchise that has 10 gaps.