It's kind of weird that my experience with it wasn't really as great as everyone else. The 2D Mario games (and SMRPG) were a huge part of my childhood. I have no idea how many hours I spent on Mario 1, 3, and World, and playing 2 across the street at a friend's house.
My friend got the game before I did, I didn't get a N64 until 1998 or so. We had fun screwing around, getting the occasional star here and there, but it wasn't really an awe-inspiring experience for us anymore than Bug! was on the Saturn. Or for me, at least.
When I got my N64 Mario was bundled with it, so naturally it was what I spent time with initially. It still didn't really click with me as a particularly amazing game. Even back then I preferred the 2D Mario games to the more open design of Mario 64 - it never really felt "Mario" to me in the way those games did. (And after Sunshine I pretty much gave up on the series, I got Galaxy on a reluctant whim due to the reviews, fortunately it was amazing)
Now fastforward to a more present-day me, older, wiser, and more capable of critiquing the game. Without the "THIS IS MINDBLOWING" nostalgia glasses, I find the game to be extremely flawed from a level design and objective standpoint. The game doesn't really get "good" until you get to the upstairs levels, and the last ones (especially Tick Tock Clock, which is amazing) are where it actually starts reaching the pedestal so many people place it on.
On a more positive note, the controls are fantastic for a game from 1996.