It must have been an amazing experience to play it back when it was released! Unfortunately, I kind of lost interest in gaming around the time of the Dreamcast (for reasons I can't recall), so didn't have the pleasure.
For me, the slow pace of the game reminds me a lot of the point and click adventure games I loved as a kid (minus the tricky puzzles, and with some fighting action thrown in). I remember around the time Yakuza 0 was released there was a lot of confusion about what kind of game it was (was it a Japanese GTA, Sleeping Dogs clone? etc). Now people just accept Yakuza as its own unique kind of game. I think it'll be the same for Shenmue 3 when it comes out too (although I don't expect it to be as successful as Yakuza 0). As an aside, I actually played my first ever Yakuza game immediately after Shenmue 2 (the original PS Yakuza), so it was interesting to be able to compare them like that.
It was really an interesting time. It was a time when something as benign as having the ability to buy a drink from a vending machine was novel as hell. I try to remind people: the year prior to Shenmue's initial release (1998) was the year of Resident Evil 2, Metal Gear Solid, Ocarina of Time. All very impressive games, but nowhere near the level of realism and detail of Shenmue, only a year later. The fact that you could rotate/move the camera was an impressive concept in 1998, much less interact with and inspect the environment. I mean hell, you can pick an orange out of the fruit bowl and twirl it around in your hands. Why? Why not.
We were also impressed by the amount of buildings you could go into. Playing stuff like RE2, we would imagine what various shops and buildings might look like inside. In Shenmue, you could simply walk in, even if there was no contextual reason for doing so--you can complete the game without even entering most places (even more so in Shenmue 2). And each one was populated with NPCs bearing unique, voiced dialogue for any point you're at in the story.
So any time someone says something about it being slow or lacking in action--a fair assessment, honestly-- I think it's important to remind them that what they're looking at with Shenmue is three things: A technical achievement, a completely self-contained bio-dome of virtual life accessible to anyone with a Dreamcast (all six of us), and a movement of art from one of the biggest visionaries in the industry.