I plan to use my next console for much more than gaming.
To be quite frank the actual gaming side isn't even what made me pre-order a next gen console. Forza 5 comes free with my pre-order and I'm not really wanting to spend any more money yet.
Having apps that I use like Xbox Music/Sky TV/Netflix require me to exit my current app to look at/start/use any other app or even go back to the dashboard.
Having to switch between inputs on my TV when switching from PC gaming/web browsing to console.
I agree with your points. I also agree that the user interface is important.
I'm planning to use my Xbox One for more than just gaming, in line with how I've changed my entertainment habits as I get older and have less free time. I listen to more music and watch more streaming vids now. Also, I would love to be able to play an Xbox Live poker game (or any other game) snapped as I use Skype/Voice chat.
One thing I hope they integrate is customized voice commands...for instance, I actually like Kinect fitness software but they take too long to boot up and start (yeah, I'm lazy). If I come home from work and say, "Xbox Fitness" it would be awesome for the system to power up, recognize me, turn the TV on, and pick up my daily workout routine where I left off with full stat tracking. That would be extremely convenient and enticing.