Can we please not pretend that anything "arcade" means "harder"... Arcades are designed to be mostly straight forward and inviting. Yeah, they ramp up the difficulty super high because they wanted to munch quarters... but home consoles alleviated the quarter munching so it's not a very frustrating experience.
Having had both a Saturn and a N64 (actually got the saturn first, my first sega console I owned, had been strictly nintendo since the NES) I can tell you there was nothing "hard" about Panzer Dragoon, Virtua Fighter, or Sega Rally compared to the games for N64 you listed (Hell, Mario Kart is as "arcade-y" as games get... which is why they made it into an arcade game the next generation )
Arcade were designed for that, sure. But they were designed to be straight-forward and inviting to those that were already adept at playing games. They didn't slowly ease you into the games world and mechanics, starring you out in safe areas, and teaching you all the basics over a period of multiple hours. It was a case of "Game Start! Oh you don't know how to play? Too bad.. GAME OVER!".
If you're actually going to try to tell me there was nothing hard about Virtua Fighter in comparison to the popular N64 selections of the time, then I can't understand the viewpoint you're coming from really, I had countless people over to play it during those years, and it always ended up the same way... mash your way up to Lion, and then get held by the AI forever, because it expects you to know what you're doing (and god help you if you got lucky, and moved on to Jeffry). Mario Kart is a relatively slow game, with tracks that are almost all trivial to drive around and has comeback mechanics built into its base system to help anyone who may not be performing well, I don't see how its very comparable at all. Mario Kart GP is a home game placed in the arcade whilst Daytona USA is an arcade game placed in the home.