Except you don't actually know which is which on your first playthrough. So that doesn't matter. Unless all you do is skip everything but the main quest, you're invariably going to be going through tons of boring ass filler content in the name of being able to claim that your game is open world, and that you'll get 100+ hours out of it. Plus, a lot of people actually want to try to complete as much of a game as they can.
I'm not sure I get what you're saying here. The question marks (I assume that's the fluff/filler we're talking about) don't hide anything important 99% of the times. Just loot, which as you rightly said is mostly useless, money, a bit of XP. The only exceptions are the Places of Power and maybe some potion recipes among said loot, but the former are hardly essential, and the latter usually can also be found elsewhere.
If one disables the question marks, they won't miss any 'relevant' content, so to speak. That's what I did, and what I always advise. Certainly makes exploration less of a checklist.