There would've been less risk to stick with a traditional model and play along with the other guys, but that's because they'd be resigning themselves to their fate and most likely be out of the console business in another generation or two. There really wasn't a choice that they had to do something transverse to the other two, as playing along left them with GameCube results. Now they could've hedged their bets both ways and made the platform easily portable to with more power
and go off in their own direction, but coming off of the GameCube, there didn't seem to be much hope in that direction so why bother.
They placed big bets but not bigger bets that could've netted them much more in hindsight.
DS was the same way. They had to do something orthogonal to PSP or they'd be swept under the waves. When faced with the choice of rolling over and dying or giving it a fight, they decided to give it a fight. 3DS price drop last year with preparation for holiday retail space coming up might've been another example. You say it was a risk what they did, but the alternative is that "marginalizing and die" scenario, which makes the risk not seem so bad. They were pretty conservative going from NES to SNES to N64 to GameCube, and it just led to lower and lower sales.
It's not so courageous to take the road less traveled, when you keep getting knocked down on the road with everyone else.
Nintendo just isn't ready to go quietly into the night. *resists urge to post Independence Day picture*