I think a lot of people, and devs, underestimate the power of the imagination filling in the blanks.
Even if you can tell the difference between a 100k poly model and a 500k, the final result in what your brain interprets is very similar.
The brain does a lot of processing of missing cues and info.
In fact, the more detail is added, the closer it is to the uncanny valley of 'it looks real but at the same time not', which could bother some.
I would bet my neck that we could have more amazing and immersive worlds, at half the polycount, and better illymination techniques and art direction.