I draw the line at art shifts because, like I said, they aren't changing the established traits of the characters. Yes, they may look different, but I can still immediately tell who Link is when I see his character. It's one of the reasons the Twilight Princess reveal worked so well. When we saw that Link riding up on horseback for the first time, we immediately knew who it was.
Nothing about a girl Link prevents the game from being a Zelda game (and I'll still play it because Zelda is my favorite series), but I think that's one of the reasons why Link shouldn't be a girl. If sexuality doesn't matter in Zelda and the gender of the main character doesn't ultimately mean anything, why go against 28 years of history and established character traits just to say "Oh hey, by the way, Link's a girl now"?
I think it's really a matter of a glass half full/empty kind of thing, only no side is necessarily wrong. I can see why people would be fine with a female Link, I just think it would make a lot more sense to make a new IP with a female character.