You realize that this was the exact reasoning behind the PS Vita, right? A handheld that plays full, no-compromise console games like Killzone, Uncharted etcetera.
Turns out, people really aren't all that interested in playing toned-down traditional console titles on a portable.
I don't think the Vita (or the PSP) are particularly good analogs for the potential success or failure of the "console games on a handheld" aspect of NX. For one, while Vita may have had "console-like" games, by and large it didn't get actual mainstream console games. While console gamers were playing Uncharted 2, Vita owners got Golden Abyss, ditto with AC:III/AC:Liberation, CoD:BLOPS/CoD:BLOPS Declassified, Resistance III/Resistance Burning Skies, etc, etc. From both technical and business perspectives, NX is in a very different place than Vita was.
From a technical perspective, the NX (from what we know) has a number of big advantages Vita didn't:
- A far bigger resolution gap with contemporary home consoles
When Vita launched in late 2011 with a 28 Gflop GPU and a 540p screen, home consoles were targeting only slightly higher resolutions with almost 10x more GPU grunt (720p and ~240 Gflops, meaning Vita had only approximately 20% of the performance per pixel). When PS4 arrived a couple of years later, it increased GPU power far more than it did resolution, dropping Vita down to just 6% of its performance on a per-pixel basis.
While people may point to PS4 Pro and Scorpio as evidence that a handheld NX couldn't possibly keep up, they seem to be forgetting that this performance boost is almost entirely consumed by a huge resolution jump, and Nintendo are being far more conservative with NX's resolution than Sony were with Vita. If the 6Tflop Scorpio is indeed going to be running games at native 4K, then a handheld running a 720p screen (nine times fewer pixels) doesn't actually need to be immensely powerful to keep up. If we're conservative and assume that NX hits 300 Gflops and we ignore FP16, even then NX would be hitting 45% of the performance per pixel of Scorpio, far higher than Vita was to its contemporaries. On the other end of the spectrum, assuming a slightly more powerful NX and full use of FP16, it's well within the realm of possibility that NX could hit 1 Tflop in handheld mode, giving it 150% of the performance per pixel of a 4K res Scorpio game, meaning it would actually be pushing the more sophisticated graphical effects of the two.
(Incidentally, my most likely scenario for NX's GPU [2x SM, 785MHz, 1.5W] with 80% of shader code running at FP16 would put the NX at almost exactly 100% ops-per-pixel parity with a 4K Scorpio.)
- A CPU on par with (or potentially exceeding) contemporary home consoles
This one's fairly self explanatory. Vita launched with a quad-core A9 CPU running at 330MHz at a time when PS3 was sporting a 3.2GHz Cell processor which was over 50x as powerful on paper. Even a cheap octo-core A53 solution for NX would put them within touching distance of Jaguar, and a combo of A53s and a few A57/A72/A73s could potentially push NX over the top (and we've heard rumours to indicate that this is indeed the case). While much of the GPU's workload scales pretty linearly with resolution, this isn't true for CPUs, which can make porting a game to a system with a much less powerful CPU extremely difficult or even impossible, something which shouldn't be an issue for NX.
- Highly scalable game engines
While multiplatform game engines were commonplace at Vita's launch, the notion of a single engine which is designed to scale from a mobile phone to a top of the line gaming PC was pretty alien. Nowadays, however, the most popular third-party engines are almost all designed to scale from systems far weaker than NX to ones far more powerful than Scorpio, and development has been consolidated to fewer and fewer engines (meaning fewer people need to do the work of actually porting game engines to NX). Where for many PS360 (and certainly PS4) games it would have been extremely technically challenging, if not impossible, to port the engine to Vita, the majority of modern game engines (even internal ones like Frostbite) already accommodate hardware like NX's, and the amount of time required to get the median PS4/XBO game up and running on NX should be orders of magnitude less than to get the median PS360 game running on Vita.
- Desktop-class GPU architecture
While the PowerVR GPU in Vita was very capable for a handheld of its time (and actually more feature rich than RSX in certain attributes), it was nonetheless a different architecture, with a different feature set, than any developers working on PS360 games would have been used to. A small number of developers with backgrounds in iOS development may have had some experience to leverage, but working within OpenGL ES and on quickly-iterated hardware wouldn't have given them much of a chance to learn how to tightly optimise to the extent typical in console development.
When it comes to NX, though, the Maxwell/Pascal architecture is not only feature-complete with Nintendo's GCN-based competitors, but unlike PowerVR 5-series, it's an architecture which is extremely well known and understood by a huge proportion of engine programmers. Every single multiplatform game engine out there effectively has to be reasonably well optimised for the architecture already, as the majority of PC gamers are using either it or one of its predecessors, and it's extremely well-documented for those engine programmers who do wish to push that optimisation even further. This also ties into the previous point, as game engines are both more scalable to NX's performance level and better optimised to its GPU architecture than was ever the case with Vita.
- Much larger game cards
When Vita launched with game cards maxing out at 4GB, PS3 was running games from 50GB Blu-Rays. Today, PS4 Pro is still limited by the same 50GB Blu-Rays, but NX's game cards will almost certainly be capable of scaling up to 64GB or even higher.
This means that the process of bringing big-budget console games to NX will be far cheaper, and hence far lower risk than it was with Vita, and will produce much better results. The big question is whether western third parties will give Nintendo the benefit of the doubt like they gave Sony with the PSP (and less so the Vita). Nintendo seems to be doing everything they need to on the technical front to keep NX viable for third parties, but it remains to be seen how successful they'll be on convincing western third parties of the business case for developing for NX.