I think a large part of it was blizzard just doesn't move very quickly and a lot of stuff is just stuck in the past. The client was kind of junky and battle.net 2.0 took forever to even come close to par with the first iteration just on messaging and usability.
They did a poor job on custom games just with discoverability which destroyed what could have been amazing as the tools were amazing and the people were talented.
The game was fun for a long while for me, but after a long time I just got bored, while modern balance changing in league or dota for instance is more rapid and less iterating. There was only so long that I could sit there practicing pool timings before I just wanted some different viable strategy because a lot of this had been mathed out (or thought to have been, but that was probably a community issue). I have other issues but I don't know enough about the actual balancing to say them confidently.
I think they did poor supporting the pro scene, which was tons of fun in the beginning.
Also, and what may have been the biggest nail in the coffin, is that this came out right before all this free to play stuff was exploding. The multiplayer client should be free. Get as many people playing as possible.A 60 dollar entrance fee is too much, compounded with the fact that they announced the fun 2 expansions at 60 dollars each... That I don't blame them for not seeing ahead of time, but reacting to it definitely should have happened. Also the mention of this stuff taking FOREVER to come out goes here. Valve is a cheetah compared to blizzard sometimes.
Also, and this is fairly asinine, but they sort of failed on a lot of the more cosmetic fun larger client meta stuff. More portraits that were easier to get, more of the insignias, more cosmetic bullshit to keep people interested. Yes I'm asking for building hats. I also don't blame them for not seeing that, but they should have reacted. A lot of people love that stuff and it can really help you keep a larger casual audience.
So it's a multi faceted issue with no one simple reason why it's floundering.I don't believe it's just cause rts isn't cool anymore, but hey that might be the reason I dunno.
I don't envy Blizzards position at all.