The main reason GCN is better than VLIW5 is because it's a simpler architecture to optimize drivers for on PC, where games are programmed upon much more abstract layers than on consoles (and also GPGPU I presume, but I don't know how a modified VLIW5 would perform in that regard).
VLIW5 in a console makes a lot of sense because EVERYTHING will be programmed towards this architecture, which is more efficient per mm^2 than GCN.
Yes, VLIW5 was rarely taken advantage of, thus the theoretical FLOPs were never really reached, because usually only 3 or 4 of those 5 units were used, this is why they moved on to VLIW4, and further improvements to this were made with GCN which gets much closer to the theoretical FLOPs.
And like you said, in a console this would be very different. A well coded game will have all 5 units in use and can rarely go unused, making it hit it's theoretical FLOPs much more often than it ever did on PC, this can also be seen somewhat with simpler tasks such as cryptography I believe, where VLIW5 performs as well as GCN per shader.
Case in point, Bitcoin mining sees HD5870 regularly hitting 440Mhash/s /w 1600SP @ 1GHz, HD 7950 hits ~510Mhash/s /w 1792SP @ 1GHz.
This is of course just one "benchmark" that uses all SPs all of the time, VLIW5 being coded for at such a low level should in theory be very similar in performance.
Another quick reason I think I'm standing on solid ground with this theory is VLIW4, the HD6970 had 1536SP, out performed the HD5870 in just about every benchmark, but has ~identical performance in bitcoin mining. HD6000 series "cut the fat" (the 5 units shrank to 4 units) to gain this performance, but consoles work in a different way than computers and coding closer to the metal allows VLIW5 to use its full potential which would put the HD5870 beyond HD6970 at the same clocks.
That is of course without taking into account other advances in the architecture like using more advance tessellation units or other small changes that were made between these cards, remember though that we have no idea how custom, if at all Wii U's GPU is. Nintendo's history points to some fairly big customization however, and I wouldn't put VLIW5's flops below GCN's in a console setting.