Playing it again reminds me of how exceptional it really is, but I feel the legacy is tarnished by two things;
1) Pacing. TP has probably the worst pacing of any of the 3D Zelda games. The tears of light quests are boring as shit, especially on replays, and the first ~1/3rd of the game is too heavy handed with tutorials and story. It simply takes far too long for the game to open up and let you do what you want to do, and I feel this is the biggest factor turning people away from replaying the game.
2) The overworld is pretty boring in the grand scheme of things. Its size makes for a lot of completely empty space and a lot of a-to-b travelling, and if not that then simply teleporting. The overworld feels a bit unfinished in my opinion, or like it wasn't used as they originally intended (perhaps a more open game).
Otherwise TP has, in my opinion, the absolute best dungeons and boss fights in the entire Zelda series. Even at their worst the dungeons are loaded with gameplay gimmicks, puzzles and varied use of items. Each dungeon having both a sub-boss and a main boss makes for two memorable unique battles each time, and though easy almost all are really creatively designed and not just "HACK SLASH FIGHT FIGHT". All the dungeons look gorgeous as well, with striking variety between them.
Its everything surrounding them that drags the game down a bit for me, mostly the going from a-to-b to get the dungeons, and the perhaps unnecessarily restricted formula and presentation in trying to make an "Ocarina of Time sequel" or whatever. Ironically, in sticking too close to the formula the 'traditional' dungeons ended up being the apex of the entire series. They feel the most focused and polished aspect of the game for sure.
Different strokes for different folks, as always, but its an excellent game through and through.