The question really has to be what OS is Nintendo going to use? Surely they arent going to build in the featureless Wii U OS or the dated 3ds OS.
Do we think there is any possibility Nintendo uses a form of Android since it already works on Tegra? This would be similar to what the Kindle tablets do .. they run a custom version of android I believe. Full blown Android Nougat uses roughly 1.5 gigs of ram on the new Pixel phone.. I know that Nintendo wouldnt have as many processes to keep going in the background so it could slim that down.
Android would be a pointless power ceiling loss and expose them to Oracle's lawyers (who are currently suing Google over Android and the API) and the GPL. I'm gonna assume it's a FreeBSD variant just like the Wii U and the PS3/4.