Support NeoGAF

[Bernkastel]

Get ready for the 'holy grail' of computer graphics

Ray tracing produces stunning lighting effects but has been too complex for computers, until now.

www.bbc.com

Interview with Xbox's Jason Ronald and Andrew Goossen, 4A Games Ben Archard, Mojang's Kasia Swica, Nvidia's Rev Lebaredian and AMD's Frank Azor on Ray Tracing in Next Gen platforms.

Ray tracing has always been the "holy grail" of computer graphics, says Jason Ronald, head of program management for the gaming console Xbox. The technique simulates a three-dimensional image by calculating each ray of light and promises stunning lighting effects with realistic reflections and shadows. The method finds where it bounces, collects information about what those objects are, then uses it to lay down a pixel and compose a scene. While the techniques have been around for a long time, "we just haven't had processing power to deliver all of that in real time", Mr Ronald says. In Hollywood, special effects have used ray tracing for a decade. For an important sequence, computers could churn overnight for a single frame. To do this for games in real time, you need to condense that to 1/60th of a second. Processing speed has now caught up to the task.

Programmable shaders started appearing around 2001. They did a much better job at 3D lighting tasks but required much more computational power. "If we put all that into one game, the most amazing processor in the world would've gone just no, it's too much," says Ben Archard, senior rendering programmer at Malta-based 4A Games, developers behind a 2019 post-apocalyptic game called Metro Exodus. There were ways around that. If a programmer wanted to simulate the hazy light coming through fog, instead of working out all the points, they could just calculate a sample of them. (These are called stochastic, statistical, or Monte Carlo approaches.)

With lockdowns around the world due to the coronavirus pandemic, the need for people to feel close while isolated "is going to accelerate" progress in technology, says Rev Lebaredian, vice president for simulation technology at Nvidia, in San Francisco. "With virtual and augmented reality, we're starting to feel like we're in the same place together," he says. So coronavirus will drive demand and progress, agrees Frank Azor, chief gaming solutions architect at AMD.

One "fiendish problem" for ray tracing has involved how shaders can call on other shaders if two rays interact, says Andrew Goossen, a technical fellow at Microsoft who works on the Xbox Series X. GPUs work on problems like rays in parallel: making parallel processes talk to each other is complex. Working out technical problems for improving ray tracing will be the main tasks "in the next five to seven years of computer graphics, at least," says Mr Ronald.

PCGamers Jarred Walton on the Unreal Engine 5 PS5 demo

In the meantime games companies will use other techniques to make games look slicker. Earlier this month Epic Games, the makers of Fortnite, released its latest game architecture, the Unreal Engine 5. It uses a combination of techniques, including a library of objects that can be imported into games as hundreds of millions of polygons, and a hierarchy of details treating large and small objects differently to save on its demands on processor resources. For most game makers such "hacks and tricks" will be good enough, says Mr Walton.

 

[Hendrick's]

I'm still of the opinion that RT is going to define next gen. It just transforms games.

 

[JareBear: Remastered]

Lumen in UE5, or other software based RT-like effects, can they render a similar image of hardware based RT but at less performance hit?

 

[Pizdetz]

What's the best example of with/without raytracing that shows tangible benefits? All of what I've seen is trivial, and in some cases the raytracing makes the scene too dark.

 

[Bernkastel]

Lumen in UE5, or other software based RT-like effects, can they render a similar image of hardware based RT but at less performance hit?

A full team of hundreds of people with around a 100 million dollar budget working on a game for more than 4 years with the R&D backing of Microsoft or Sony can.

 

[Optimistic Gamer]

What's the best example of with/without raytracing that shows tangible benefits? All of what I've seen is trivial, and in some cases the raytracing makes the scene too dark.

 

[Bernkastel]

What's the best example of with/without raytracing that shows tangible benefits? All of what I've seen is trivial, and in some cases the raytracing makes the scene too dark.

He explains it really well

 

[Bernkastel]

Speaking of next gen Ray Tracing

 

[JOEVIAL]

I am super curious what CONTROL uses before raytracing... because that game is still beautiful without RTX on. Reflections and the ambient occlusion lighting look fantastic in that game.

But yeah, games will be prettier for sure with some excellent artists behind them and RTX on, but I wonder how many people will stop to notice these details? In motion it can be hard to tell the difference.

I for one definitely stop to notice all the small details and things, especially in a game like CONTROL. Loved my time with it.

 

[Guilty_AI]

What's the best example of with/without raytracing that shows tangible benefits? All of what I've seen is trivial, and in some cases the raytracing makes the scene too dark.

RT benefits more the developer side of things rather than the players side.
Its much easier to implement than traditional lighting techniques and has better results.

 

[Endless Fluff]

I wonder how many people will stop to notice these details?

They won't notice them at first, but they will notice them when they get used to it and then they go back to an old game that doesn't have it. Just like you instantly see the lack of Ambient Occlusion and Global Illumination (really small things that transform a scene) in PS2 era games. You'll notice that something looks off.

 

[Miles708]

RT benefits more the developer side of things rather than the players side.
Its much easier to implement than traditional lighting techniques and has better results.

That's how I've understood it too, but I think the main point is that it's a somewhat marginal improvement over a massive performance hit.
Even on next-gen hardware, it's always a balancing act to see if it's really worth the performance cost.

The solution showed in the UE5 showcase, while maybe a bit less accurate, could be the right path to follow. It's not necessary to replicate reality 1:1, it's enough to keep the general convincing behaviour, while simplifying it as much as possible.

 

[Orta]

If the next gen consoles provide ray tracing on tap without the games taking a hit I'm expecting quite a few (below par especially) developers to implement it in the tackiest, shiniest, most lurid and overblown, non-subtle in-your-face way possible just for the sake of having it in their game worlds

It could be like n64 fogging all over again.

 

[FearlessBelgian]

Useless waste of graphic resources ... Especially for consoles ...

 

[Kuranghi]

He explains it really well

"Explains it really well" might a stretch. This feels like the level of answer you would get if you asked your "gaming friend" what RT was between rounds at a pub quiz. Sorry if that sounds harsh but just seems like you were being sarcastic when you said that because he gives such a basic answer to the question, but its not even a simple answer for laymen, its just a kinda poor description of it.

Is he a community guy or an actual dev who works on the engine? If the former obviously I wouldn't expect him to be an expert on that sort of thing.

I AM super excited for Ray-Tracing though so thanks for making this thread, who knows what we'll see in the next gen!

 

[JareBear: Remastered]

If the next gen consoles provide ray tracing on tap without the games taking a hit I'm expecting quite a few (below par especially) developers to implement it in the tackiest, shiniest, most lurid and overblown, non-subtle in-your-face way possible just for the sake of having it in their game worlds

I think you may be right. Get ready for the shiniest trees and shiniest grass ever.

But, on the flip side, when done well (we demonstrated recently) it really does look like it can be a game changer.

I mean, with DLSS 2.0 I can use RT in Control and it looks GREAT. Not sure how Control's RT compares to what we will get next-gen

Speaking of next gen Ray Tracing

The work that single dev added to Bright Memory from the initial Steam release seems really nice. RTX and sharper textures, it looks like. Really pops in that vid

 

[GymWolf]

I am super curious what CONTROL uses before raytracing... because that game is still beautiful without RTX on. Reflections and the ambient occlusion lighting look fantastic in that game.

But yeah, games will be prettier for sure with some excellent artists behind them and RTX on, but I wonder how many people will stop to notice these details? In motion it can be hard to tell the difference.

I for one definitely stop to notice all the small details and things, especially in a game like CONTROL. Loved my time with it.

i tried rtx on control but after 10 min i switched to better resolution\framerate without rtx, but i'm not really a shadow\reflection guy.

 

[Optimistic Gamer]

But yeah, games will be prettier for sure with some excellent artists behind them and RTX on, but I wonder how many people will stop to notice these details? In motion it can be hard to tell the difference.

I for one definitely stop to notice all the small details and things, especially in a game like CONTROL. Loved my time with it.

It's not about seeing the difference but feeling it. Especially effects like ambient occlusion have shown that many effects, while not obvious to laymen, can make all the difference when it comes to immersion. This is one of these technologies where people also need to have some trust in designers I feel like.

 

[Bernkastel]

"Explains it really well" might a stretch. This feels like the level of answer you would get if you asked your "gaming friend" what RT was between rounds at a pub quiz. Sorry if that sounds harsh but just seems like you were being sarcastic when you said that because he gives such a basic answer to the question, but its not even a simple answer for laymen, its just a kinda poor description of it.

Is he a community guy or an actual dev who works on the engine? If the former obviously I wouldn't expect him to be an expert on that sort of thing.

I AM super excited for Ray-Tracing though so thanks for making this thread, who knows what we'll see in the next gen!

He is the Technical Art Director.

 

[Kuranghi]

He is the Technical Art Director.

Aye fair enough then, he's not exactly an engineer or anything, just making sure that wasn't the lead engineer or something lol, then I'd be worried. I mean this next part optimistically btw: I'm sure you can find a better video of someone explaining RT in a non complex way.

I'd be willing to keep an eye out for one and I'll send it your way if/when I see it.

edit - Ooo it was freaky when the quote changed after I hit post, thought I had a brain explosion.

 

[StormCell]

That's how I've understood it too, but I think the main point is that it's a somewhat marginal improvement over a massive performance hit.
Even on next-gen hardware, it's always a balancing act to see if it's really worth the performance cost.

The solution showed in the UE5 showcase, while maybe a bit less accurate, could be the right path to follow. It's not necessary to replicate reality 1:1, it's enough to keep the general convincing behaviour, while simplifying it as much as possible.

I don't really see it like that. I feel like photo-realistic high detail graphics have been stuck in the uncanny valley void for several years, and I didn't know what that difference was until the RT demo videos shed some light on the issue. The truth is that you can't track down all the tiny details you're missing under the current lighting schemes. Your eyes see all the details and you tell yourself this is as realistic as it gets, but things will always feel slightly off. Eventually you begin to see some of the missing details, like in Spider-man. It was supremely disappointing not to get a nice mirror reflection of Spidey off the buildings.

Still, I'm in agreement that until hardware is capable of implementing RT on top of the best looking games today, I'm not really interested in seeing a step back across the board except for superior lighting.

 

[zeomax]

If the next gen consoles provide ray tracing on tap without the games taking a hit I'm expecting quite a few (below par especially) developers to implement it in the tackiest, shiniest, most lurid and overblown, non-subtle in-your-face way possible just for the sake of having it in their game worlds

It could be like n64 fogging all over again.

Or the bloom era during the PS360 time

 

[hyperbertha]

pretty bad example imo. Lots of reflections that are also partially present in the non raytraced versions and nothing else. For tangible differences games need to be fully path traced whereas most of these AAA games are only partially raytraced, with ambient occlusion and reflections.. I think Minecraft and the marbles demo is the only example of full raytracing currently currently?

 

[Wandering Leopard]

Ray tracing will be a big deal if you are PC gamer with a card around the level of an Ampere/RDN2 (which will be operating at a higher power budget).

I'm not expecting much from next gen consoles, maybe hybrid implementations/low use in AAA games and high usage in smaller games like this:

 

[Optimistic Gamer]

pretty bad example imo. Lots of reflections that are also partially present in the non raytraced versions and nothing else. For tangible differences games need to be fully path traced whereas most of these AAA games are only partially raytraced, with ambient occlusion and reflections.. I think Minecraft and the marbles demo is the only example of full raytracing currently currently?

Yes but they are also not "tangible" examples as one has a very unique style and the other doesn't run on your home-machine at all.

 

[JareBear: Remastered]

Ray tracing will be a big deal if you are PC gamer with a card around the level of an Ampere/RDN2 (which will be operating at a higher power budget).

I'm not expecting much from next gen consoles, maybe hybrid implementations/low use in AAA games and high usage in smaller games like this:

In one of the recent DF vids, I think after the XsX third party stream, they were saying they expect most console games to use partial RT, not full blown RT everywhere, which seems realisic, unless we get mid-gen refreshes with geatly increased RT ability

 

[nikolino840]

"Explains it really well" might a stretch. This feels like the level of answer you would get if you asked your "gaming friend" what RT was between rounds at a pub quiz. Sorry if that sounds harsh but just seems like you were being sarcastic when you said that because he gives such a basic answer to the question, but its not even a simple answer for laymen, its just a kinda poor description of it.

Is he a community guy or an actual dev who works on the engine? If the former obviously I wouldn't expect him to be an expert on that sort of thing.

I AM super excited for Ray-Tracing though so thanks for making this thread, who knows what we'll see in the next gen!

Well if you want more info

Ray Tracing: Graphics for the Masses

A practical introduction to ray tracing 3D graphics for readers not familiar with graphics

www.cs.unc.edu

 

[Bartski]

TBH these are some pretty good examples of why I think it's completely not worth it.
It looks nice don't get me wrong. I understand some form of it will soon be a standard, even a necessity for next-gen visuals. To me it's probably the least exciting thing in terms of video game tech moving forward in years and I'm kind of surprised how much perfectly reflected highlight it's getting. I'd really prefer all this power going elsewhere but I accept it and I hope physics will eventually catch up

 

[nikolino840]

TBH these are some pretty good examples of why I think it's completely not worth it.
It looks nice don't get me wrong. I understand some form of it will soon be a standard, even a necessity for next-gen visuals. To me it's probably the least exciting thing in terms of video game tech moving forward in years and I'm kind of surprised how much perfectly reflected highlight it's getting. I'd really prefer all this power going elsewhere but I accept it and I hope physics will eventually catch up

I think are both powerful to keep both,or at least the minimum setting of raytracing in a PC

 

[Wandering Leopard]

In one of the recent DF vids, I think after the XsX third party stream, they were saying they expect most console games to use partial RT, not full blown RT everywhere, which seems realisic, unless we get mid-gen refreshes with geatly increased RT ability

I also look forward to more devs completely moving over and dropping todays techniques, the only dev so far that have said they're doing this 4A games - https://screenrant.com/metro-exodus-4a-games-next-gen-ray-tracing/

 

[Sethbacca]

Watching this and knowing what the performance hit involved is when implemented in hardware, I feel like the UE5 hybrid option is likely a better approach until the hardware is capable of performing in such a way that it doesn't drag everything down. Yeah, it looks better no question, but just not sure it's worth the performance hit is all.

 

[Slings and Arrows]

I don't see how RT is going to add anything to console gaming other than a new lick of paint at the expensive of better physics, AI and destruction.

Inb4 a new technology comes along that looks just as good at half the performance cost.

 

[V4skunk]

I don't see how RT is going to add anything to console gaming other than a new lick of paint at the expensive of better physics, AI and destruction.

Inb4 a new technology comes along that looks just as good at half the performance cost.

RT is the holy grail.
New tech will come but it will only make RT more achievable.

 

[Danjin44]

RT is the holy grail.
New tech will come but it will only make RT more achievable.

How so? will change how we play games like switching from 2D to 3D? Will the game with RT play different than same game without RT?

 

[V4skunk]

How so? will change how we play games like switching from 2D to 3D? Will the game with RT play different than same game without RT?

You don't have a clue.
How do you think film studios make CGI?

 

[Soodanim]

We get 120hz support In consoles and everyone spends the resources on Ray Tracing implementation to suit that one mirror scene and the game runs at 35fps instead! (I joke, but it will happen at least once.)

I see it as very pretty, but probably not worth it half the time. I think Lumen does more for a scene than RT in most cases.

 

[Danjin44]

You don't have a clue.
How do you think film studios make CGI?

Again how is that effect the actual gameplay!? People who make CGI don't have to worry about actual game is running like combat, AI or even running high FPS or player controlling the camera or even their animation will look good in every direction.

 

[Pejo]

Man, the RTX looks good in this for some of the surfaces, but for others not so much. It seems like the medium reflective surfaces or wood with a sheen look fantastic though.

Also, I haven't played this, but it feels like the main character is redhead Hilary Clinton. What an awful character design.

 

[Endless Fluff]

Again how is that effect the actual gameplay!? People who make CGI don't have to worry about actual game is running like combat, AI or even running high FPS or player controlling the camera or even their animation will look good in every direction.

The topic is "holy grail of computer graphics" as in rendering, not "holy grail of gameplay and animations".

Also it WILL change how we play games. Ever been in an infinite mirror maze? Such a setting is currently impossible in games and I won't see it becoming possible with next gen either, but someday.
Another example? Proper mirrors for horror games. Like... having a mirror to peek around corners? Using a knife/some steel for that? Realistic reflections can be used for many things.

 

[Danjin44]

The topic is "holy grail of computer graphics" as in rendering, not "holy grail of gameplay and animations".

Also it WILL change how we play games. Ever been in an infinite mirror maze? Such a setting is currently impossible in games and I won't see it becoming possible with next gen either, but someday.
Another example? Proper mirrors for horror games. Like... having a mirror to peek around corners? Using a knife/some steel for that? Realistic reflections can be used for many things.

I dont think Sakura Wars has RT but that game proper mirrors even without RT. Hack! Even Luigi's Mansion has proper mirror and you do some of the puzzles using mirror's reflection.

 

[Endless Fluff]

I dont think Sakura Wars has RT but that game proper mirrors even without RT.

And how many of them mirrors do you use for gameplay reasons? Your post is either disingenuous or you don't understand my point.

We haven't seen an infinite mirror maze in a game yet, we also haven't seen games taking advantage of reflective surfaces in gameplay, yet, which would be perfect for horror games. The mirrors we got in games so far have been extremely limited, the best ones are typically in racing games but even there you get no visibility, this will also change with RT reflections. Another gameplay improving thing.

 

[NeoGiffer]

 

[hyperbertha]

Again how is that effect the actual gameplay!? People who make CGI don't have to worry about actual game is running like combat, AI or even running high FPS or player controlling the camera or even their animation will look good in every direction.

Its just the holy grail of game graphics. To reach CGI levels of graphics we need ray tracing. What you are referring to is a real shift in game design that we haven't had in games for decades. Something like this:





The holy grail of gaming is imo in game design improvements that lead to much higher levels of interactivity in games than seen currently. Certain indie devs seem to be making strides in that area, though AAA studios generally stay away due to 'risk'.

 

[Slings and Arrows]

RT is the holy grail.
New tech will come but it will only make RT more achievable.

Holy grail? Better IQ?

Where's the physics, destruction and clever AI? All three have taken a back seat and in terms of AI have become worse after every generation.

RT is an excuse for publishers and developers to sell more of the same shitty games under the guise of "its pretty". And no, this isn't the same as HD, 4k etc.

Red Factions destruction was 25 years ago.

Battle field bad company 2 was 10 years ago.

Yet, were supposed to believe that RT is going to move gaming forward when we've left the building blocks and the very foundations of games in the past?

RT is power of the cloud, infinite endings and secret sauce levels of pointless bullshit that is pushed on us, the consumer, and we have to swallow it.

 

[Kuranghi]

And how many of them mirrors do you use for gameplay reasons? Your post is either disingenuous or you don't understand my point.

We haven't seen an infinite mirror maze in a game yet, we also haven't seen games taking advantage of reflective surfaces in gameplay, yet, which would be perfect for horror games.

I get what you are trying to say about the gameplay implications of RT but you kinda made yourself your silly here, I'm guessing you didn't play Luigi's Mansion 3?

Its both a horror game and you often use mirrors for gameplay reasons. Not an "infinite hallway of mirrors" or anything but it literally is what you said doesn't exist lol. You use mirrors to see the other side of the level reflected and find secrets but you also fight enemies you can only see in the reflection as well.

Hitman 2018 also has working mirrors, the AI will react to seeing you behind them in the mirror.

edit - its funny to think of LM as a horror game, prob better to call it "spooky" lol

 

[deriks]

I'm still of the opinion that RT is going to define next gen. It just transforms games.

Same

 

[jackets with pockets]

Or the bloom era during the PS360 time

 

[Endless Fluff]

I get what you are trying to say about the gameplay implications of RT but you kinda made yourself your silly here, I'm guessing you didn't play Luigi's Mansion 3?

Its both a horror game and you often use mirrors for gameplay reasons. Not an "infinite hallway of mirrors" or anything but it literally is what you said doesn't exist lol. You use mirrors to see the other side of the level reflected and find secrets but you also fight enemies you can only see in the reflection as well.

Hitman 2018 also has working mirrors, the AI will react to seeing you behind them in the mirror.

No, I haven't played Luigi's Mansion, but I've also said (which you've cut out from my quote) that the mirrors we've got in games so far have extremely limited use and they're not really what I have in mind when I talk about what RT reflections mean for gameplay, but you are completely right about the Hitman reflections. I totally forgot about that.

My point still stands, though, RT can change and elevate that gameplay of Luigi's Mansion and Hitman by expanding it, instead of using fixed mirrors (or windows) you have dynamic ones. In your hand. Think about shadows and how they evolved and became part of gameplay features. Doom 3? Splinter Cell? The whole atmosphere changed when we could cast shadows ourselfs instead of relying on pre-baked shadows.

I expect to see enemy behavior from Hitman in many more games in the future, just like enemies react to shadows in some games... but I don't expect all games to change in a dramatic way.

 

[Bolivar687]

This is dumb.

 

[JareBear: Remastered]

This is dumb.