In all seriousness, I'm not saying that what you are proposing is ridiculous. Not at all. You can fake a lot and get some really good results
Typically every single shot is custom tailored in many cases in most games we see. Personally, as I have come to term's with this years ago, I take no displeasure in alerting the standard consumer that the artist making their most sought after Triple A Videogame Title - even if they have created a graphical marvel - still in fact did not utilize all the tool's and or techniques at their disposal to deliver the best possible. And that is not a case of optimization that is an issue typically constrained to minimize time, effort and cost.
In all seriousness, as the industry continues to move forward - the techniques leveraged will in fact be less reliant on Hardware Accelerated features as time goes on.
We are probably one generation away from machine learning replacing in fact all art rendered on screen with higher optimized variant's of asset's. Including Ray Traced Variant's.
DirectML, as cited by Microsoft may end up doing that for this current generation. We have not yet seen what this technique can substantiate.
But currently there are a myriad of techniques the artist in fact does not worry about that would accommodate Performance AND Visuals if applied directly.
Custom Culling of 3d Assets on screen has leveraged remarkable benefit's due to the fact that 3d engines currently only apply culling on the fly with a less than accurate fuzzy math - unless more coding is added during the development cycle.... and even then those result's would not be nearly as optimal as an asset Culled/Crafted by hand from the artist. Meaning in fact, that asset would need a new LOD option to simply swap out culled animation models.
Custom culling of asset's in game isn't really a technique artist's or dev's consider, nor is it something the industry utilizes. It's a very consuming way to budget for performance. But I can see machine learning applying a variation of culling to asset's to gain performance - that in turn exceeds what engines do currently. Software Optimization's like this will come and then persist.
For the non informed. Culling is a method of taking a 3d object, and slicing away the geometry you can not physically see. Example - Take a character, and cut away literally all polygon's not seen.
The back of the characters shirt, the back of the characters legs, everything that is not visible by the camera. Do it again for the front, side, top view... angles....
Engines actually do this now, poorly.
Unless more coding is added to help alleviate the fact that your dev team may not be using standard creation method's when building a game to begin with.
And then still... the result's aren't terribly great. Machine learning applied to this technique will leverage far more performance. And I do, personally, when that option arrives - consider it a Software fix.
We will still have plenty of option's as artists, to utilize software or hardware defined method's. What you are saying in fact only correlate's with what I cited in my latest post - artist's are capable of delivering result's that match ray tracing 100% through meticulous and nuanced refinement of source maps/cubemaps/ shadow maps/spec maps and 3d asset's. Whether those techniques are time consuming or the development tool's cost more - does not mean these technique's could not be applied to deliver large performance gains and with matching result's where Ray Tracing might otherwise be enabled.
But yes, Ray Tracing in hardware is certainly nothing to scoff at - but remember consumer - when a game offer's visual's but with a reduction in raytracing utilizations (some games may offer only Ray Traced lighting, other's may offer a full suite of Hardware Accelerated Ray Tracing but at modest application) - there were method's to deliver a full fledged, insanely Ray Traced product without sacrificing performance and Ray Traced Visuals - only had the artist's involved invested enough time to deliver it - and you would have never known which part's had Ray Tracing disabled.
That is where you, as the standard consumer wanting the best visual's possible - are at a loss. Particularly when you add in the fact that there are many examples of photorealism techniques which I offered on display in this thread, that have oft been ignored for nearly a decade now in most games seeking to deliver Realism. The worst culprit being underutilization of High Resolution photo accurate textures.
Something that's been possible for nearly 2 decades now. But we still see artist's offering painterly or simply blurred, less accurate Texture variations in games.