I think games in the genre still have plenty of potential to sell, it's always going to be a niche genre, but the three games you listed that suffered from poor sales all had outside obstacles hurting them in addition to the fact that they just inherently sell less than than other genres.
DmC had a massive controversy and fanbase divide that surrounded it, turning off longtime fans and likely a fair few other people in the process with all the vitriol that surrounded it. The original version of Ninja Gaiden 3 was complete, pure garbage and even though the rerelease fixed many of its issues it still had a lot of problems from the poor base game it was built from. And Bayonetta 2 is an exclusive on a console in which even third party titles that are highly successful on other platforms suffer in sales.
For these games to succeed they need to be widely available (a game like this needs as many sales as possible, and being restricted to one platform limits that greatly), they need to know their target audience (fans of technically proficient, mechanically deep games with high skill ceilings) and they need to provide the depth that their most loyal fans want while also being accessible enough (Wonderful 101 is probably the worst offender at having a high barrier to entry, while Bayonetta is much more friendly to new players), and having enough spectacle in a single playthrough to wow the people who will only run through the game once (no more half-a-game DMC4 situations).
And probably most important of all, and least likely to happen, developers/publishers need to have reasonable expectations. A properly developed DMC5 can easily reach Capcom's 2 million sales expectation. But that series probably has the most clout in this genre besides God of War, and very few other games should expect those kind of sales.
