I think the problem stems from two interconnected issues:
1) Nintendo never really learned how to earn third party support with good competition present. Nintendo had that support in the NES and SNES days because it was too big a part of the market for any console developer to ignore. It was Nintendo's game back then. As soon as real competition showed up the developers rushed to it en mass like all the air getting sucked out of a punctured airplane. Nintendo hasn't really figured out how to adapt to this, partly due to its other problem.
Not true at all. Nintendo had competition since the SNES days. Yes, they dominated with iron fist in the NES days, but SNES they had to struggle. Genesis was dominating on the US market and they only managed to turn around on it's late life. In Japan, SNES came out two years after the Genesis and PC-Engine, so there were alternatives, if they were keen to leave Nintendo. Still Japanese third-parties decided to stick with Nintendo, instead. In US, despite considerable Genesis domination, SNES still had almost every single considerable third-party support from the time.
It was from the N64 onward that Nintendo started their "shoot themselves on their foot" hardware decisions that scared away third-parties.
2) Nintendo kind of has a core philosophical disagreement with all the big western publishers. I feel like Nintendo doesn't really care for the way big console games have been made ever since around the time Final Fantasy VII came on the scene with its cinematic presentation and huge marketing campaign based on the story and cut scenes. The big western publishers have built their business off those kinds of games made specifically for the western young adult male demographic and have made it their niche. I think the main decisions makers at Nintendo simply don't especially enjoy those kinds of games and prefer things to be like how console/arcade gaming was in the 80's and early 90's. That is, games based almost entirely on mechanical gameplay made for all audiences.
Furthermore, the west wants increasingly powerful hardware to allow them to make bigger and prettier games, Sony having gone along with that. Nintendo on the other hand primarily just cares about making more purely mechanical games while trying to create more innovative control interfaces. Mind you, the N64 was still powerful hardware for its day, but Nintendo made it so because it needed the advanced 3D graphics to make its own ideas like Mario 64 a reality. The Gamecube is probably the sole case of Nintendo deliberately keeping up with the joneses, and people at Nintendo have already expressed their displeasure at what happened during those years. I know at least one third party company started feeling down about the Wii as soon as Nintendo revealed to publishers it wanted to make the console low-wattage for the benefit of the Japanese market, which is part of why it was underpowered. The same goes for the Wii U, though it was a half-attempt to get the hardcore back.
I'm really starting to think the big western publishers are essentially lost to Nintendo for these reasons. It's not even just the ones the OP mentioned that eventually drifted away from Nintendo. The ones Opiate mentioned, along with companies like Bethesda or Epic -- the old PC guard, have never had any working relationship with Nintendo. Valve, Bethesda, and Epic I believe have never shipped games for Nintendo hardware. In Blizzard's case I think there's just the SNES version of Lost Vikings and the StarCraft N64 port. id still has some willingness to pay attention to Nintendo but that's about it. I think Nintendo would be prudent to further strengthen ties to Japanese developers and try really hard to get support from mobile and indie developers. All those groups seem to make games more in line with Nintendo's philosophy.
Nintendo had a western friendly approach on their SNES and N64 days, as they made several publishing and second-party deals with many western developers attending to appeal to this audience. DMA Design, Paradigm, Angel Studios, Bits Studios, Hardware Creation, Left Field, Silicon Knights, Rare, Factor 5, Mass Media, Looking Glass Software, H20, Saffire, Midway, Retro... they were all riding Nintendo's boat and investing into the western crowd that Nintendo's first-party titles couldn't properly manage to appeal. They managed to succeed at that, because SNES and N64 had, both, a solid Teen-Mature crowd. That's why big third-parties at time like Acclaim, THQ and Midway brought several big, successful exclusive titles for the sytem. Turok? WCW/WWF series? Star Wars? They were there.
Shikamaru Ninja made a great post on a thread last year about how was NoA back in the day:
From 1990-2000. Nintendo of America had production and management autonomy from Japan. NOA basically culminated its own production team, along a few co-designers, and started funding and producing games with developers.
DMA Design: Uni Racers, Body Harvest (Nintendo dropped it in 1997, Midway took it)
Angel Studios: Ken Griffey Baseball, Buggie Boogie (canceled)
Bits Studios: Warlocked, Riqa (canceled)
Rare: Donkey Kong Country, Killer Instinct, Goldeneye 007, Perfect Dark
Software Creations: Ken Griffey Baseball, Tin Star
Silicon Knights: Eternal Darkness (N64 version)
Left Field Productions: Kobey Bryant in NBA Courtside, Excitebike 64
Looking Glass Studio: Mini Racers (canceled)
Mass Media: Star Craft 64
H20: Tetrisphere
Saffire Corp: Nester's Funky Bowling, James Bond 007
Midway: Cruisn Series
Nintendo of America also procured the Ken Griffey and MLBPA license, NHL License, Kobe Bryant and NBA license, PGA license, Disney license, James Bond license, StarCraft license. Star Wars Episode I license. They were producing their own first-party games separate from Nintendo of Japan.
That all changed when Iwata transitioned from Global Marketing Chief to President. NOA Production was killed, and Nintendo of Japan's SPD Department took over all Western development (Star Fox Adventures, Geist, Eternal Darkness GC).
Henry Sterchi, Brian Ullrich, Ken Lobb, Ed Ridgeway, Jeff Hutt, Faran Thomason, and the whole crew left NOA to Microsoft and other developers. Since then, we've seen the Western model we have today. Western developers reporting directly to Japanese management, and pretty much making B/C sequels to Nintendo IPs.
And this is an interview with Peter Main, circa 2000, acknowledging how important was western support for Nintendo systems.
http://www.ign.com/articles/2000/05/15/the-main-man-talks-dolphin
Peter Main said:
Nintendo’s Peter Main: “We continue, let me say, to have the utmost respect for the third-party world and its potential to be a very important part of our program going forward. However, in recent times a number of things have happened beyond that. Brand loyalty, by virtue of games being available on every platform, has somewhat reduced since the NES days when we did have platform exclusivity.
We know that core to our business are the franchise characters that we own. We know the limitations of our own development skills. In that line, we set out to find partners who could further build the brand identity and personality with product that would appear exclusively on our platform.”
We’re not a cash poor company, so bringing financial resources to groups that show tremendous creativity and development potential makes good sense. The second-parties have become very logical extensions of what we do so well with our group in Japan and NST now in America. And given the rate of change with new platforms, and what essentially in engineering terms becomes fast-track projects, in order to make them happen in the least amount of time, you need to bring people into your tent long before you’re finished. That is better done with internal partners than external ones.
That all said and done, we think it helps us, especially in our next go-around, come to market with more product, and more diverse product, created by people who didn’t have to wait for final tools to be made available, and instead could do it on the fly with us as we develop the components.
But again, Nintendo has in no way, shape or form turned its back in licensees and third-party developers. There are a good number of superb people that we look forward to having even more active involvement in the years ahead.”
This is what Nintendo needs to restore and bring back to their track. Something Iwata's short-sighted management disassembled and had negative impacts on Nintendo, as they not only lost significant market share to Microsoft, which managed to grab the N64's FPS crowd, for example, as burned bridges with third-parties and lost their western appeal, restricting their audience.
Going forward, I wouldn't be surprised if the primary base for the NX platform was another underpowered machine that ends up mostly just attracting Japanese third parties and maybe some mobile developers if Nintendo is lucky and smart. Maybe the potential for more form factors could allow Nintendo to make a "super powerful" NX machine but that alone wouldn't attract the big western publishers.
That's what everyone is affraid to happen and will drive them to an inevitable failure, as this direction has proven dry already, given Wii's premature death and Wii U's failure. Gamers and third-parties aren't buying the "underpowered hardware with gimmicky controller" plan anymore.
To get them back I think Nintendo would have to:
1) Completely change how it deals with other companies, probably talking to them about new hardware while it's still in the planning stages to see what they want.
2) Be more willing to push and accommodate blockbuster games for the 18-35 western male demographic.
3) Give up on the Japanese console gaming market like Sony did. Or at least don't treat it like priority 1 anymore.
Most importantly and overall though, Nintendo needs to determine whether it's going to be a very Japanese company or start acting like an international company.
Agree with 1 and 2, but not with 3. They might keep their japanese appeal if they want, but not at expense of western appeal and support like their current japanese heavily direction is making. Totally abandoning Japan wouldn't be a necessity in order to grab western attention.