That being said, if we want to make a case for actual worst - I think the original might kind of fit the bill. I don't mean it's a bad game - it was absolutely groundbreaking, spellbinding, etc etc - but in terms of design it's by far the most obtuse and unintuitive. That's understandable since it's the first in a now well-established series. They were trying new things and the series got refined as it went on, but seriously, the original Zelda is rough if you strip it of historical context. Which, to be fair - is how it should be, considering it's the very first one.
Ooh, this is gonna make you a few enemies. But I totally agree. That and Zelda 2. Both games are rough, primitive and clunky (2 is also just straight up badly designed). There's plenty of other Zeldas I also dislike, like nearly all the handheld games (especially those garbage DS games), but even those I'd rather play than the two NES games. I just can't.