Today's hard mode is yesterday's normal mode, so I only go with hard difficulty.
For genres that I dont play often, I will usually go with normal, but most of the time I will ramp it up to hard after a few dozen minutes.
But tbh it depends on the tuning. Some games have TERRIBLE difficulty tuning where hard mode equals being two shot by mobs that have way too much health. This is just bad design.