Those games could work with touch-only controls. I may not prefer it but they would work fine. The problem is that Nintendo uses those types of games to fill out their release schedule and make their own hardware sustainable. It also makes their library well-rounded with games from all genres. I think the fear is that those games will succeed on mobile, making ports for their dedicated hardware pointless, and perhaps the smaller library of games and lack of focus on their own hardware will eventually spell it's demise. What then happens to the types of games that aren't suited for mobile? The ones that NEED buttons? I don't think carrying around a controller is an acceptable response, since not wanting to carry something extra around is the whole argument against dedicated portables in the first place.
For those who are upset, it's kind of like being an only child who is suddenly told that their parents are expecting a new baby. They don't see themselves as mobile gamers so, like the child, in their view, this new baby can only divert attention and resources away from them. It may turn out that Nintendo still loves them as much as ever, but at this moment, it feels like something is coming that will take their love away.