• Hey Guest. Check out your NeoGAF Wrapped 2025 results here!

EduGAF: fp16 precision through pictures

That's actually really confusing to me, and I already had a loose understanding of the difference between the two.
My understanding is that FP32 is not "a number with 31 decimals" nor is FP16 "a number with 15 decimals".

Someone please correct me if I'm wrong, but my understanding was that - as the name implies - floating point numbers use a fixed number of bits and, basically, the decimal moves places in the numbers those bits represent.
So for very small numbers, you have a lot of precision after the decimal; e.g. 1.23456789 while larger numbers have less precision; e.g. 123456.789

Note: the above is not an accurate representation, only used to simplify the explanation for people that haven't dealt with this sort of thing before.
I just hope that's not another "simplification" which actually complicates matters.

From the horse's mouth so to speak:

half-precision-floating-point-fp16.jpg


v8sc.jpg


Really, it's just that FP16 is trading precision for performance.
There are times when that additional precision is not going to be necessary, and should speed things up without affecting image quality.
Unfortunately I suspect that it's going to be used improperly at times, and that will hurt image quality.
It may affect image quality but if you won't be able to notice it it should be considered a valid performance optimization. On the other hand, when properly used, it should produce the exact same result as FP32 but noticeably faster (not even close to 2X though).

And as we're moving to HDR, I would have thought that games would be starting to focus on using more precision, rather than less, but I'm sure there are places you can cut the precision without affecting the end result visually.
I do wonder how much of an overall speed-up it's going to be. It's certainly not going to double game performance for example.
HDR10 is using 10 bits per channel, HDR12 is using 12 bits, FP16 is more than enough for final image representation in both of these formats. Precision sensitive parts of code which may affect final image quality will still be done in FP32.

We're supposedly only a couple of weeks from Volta unveil and I expect that there will be even more talk on mixed precision during this. FP16 and Int8 are highly usable in machine learning.
 
Someone please correct me if I'm wrong, but my understanding was that - as the name implies - floating point numbers use a fixed number of bits and, basically, the decimal moves places in the numbers those bits represent.
So for very small numbers, you have a lot of precision after the decimal; e.g. 1.23456789 while larger numbers have less precision; e.g. 123456.789
That's the result, yes. Floating point numbers are calculated from different parts, the sign (+/-), the significand (integer number) and the exponent (the position of the point). The allocation of bits for each differ depending on the total bits available. Wikipedia has a handy table: https://en.wikipedia.org/wiki/Floating-point_arithmetic#Internal_representation
 
Unfortunately we probably won't see too much use outside of checkerboarding since they can't use fp16 on PS4 vanilla

Does the OG PS4 not support fp16 at all or it's just that it doesn't get the double performance boost the Pro has?

If it's the latter then using fp16 in areas where less precision is required wouldn't make any difference on the OG PS4, it would run it the same speed as fp32 - but - that same code an a pro would receive a small boost?
 
Does the OG PS4 not support fp16 at all or it's just that it doesn't get the double performance boost the Pro has?

If it's the latter then using fp16 in areas where less precision is required wouldn't make any difference on the OG PS4, it would run it the same speed as fp32 - but - that same code an a pro would receive a small boost?
I can't say for sure now (I'm on mobile) but I think GCN 1.0 (PS4/XB1) runs FP16 slower than FP32 because it is emulated.

GCN 4.0 (Polaris) introduced native FP16 support as same speed of FP32.

If that is true then using FP16 on PS4/XB1 Will break performance... for Scorpio case it will run fine like FP32.
 
I can't say for sure now (I'm on mobile) but I think GCN 1.0 (PS4/XB1) runs FP16 slower than FP32 because it is emulated.

GCN 4.0 (Polaris) introduced native FP16 support as same speed of FP32.

If that is true then using FP16 on PS4/XB1 Will break performance... for Scorpio case it will run fine like FP32.

GCN1 and GCN2 doesn't run at FP16 at all because they don't support it. You can emulate FP16 in FP32 math but since you'll be doing this manually you'll likely be slower than just using FP32, yes.

GCN3 introduced native FP16 support and packing of FP16 registers. GCN4 didn't change anything here I think.

Got to add that compilers will simply strip half precision hints from code targeted at platforms which don't support FP16 - and that's all there is to run the same code on OG PS4/Xbox One/Scorpio. The really tricky part is to find proper places to use partial precision in your high level shader code.
 
GCN1 and GCN2 doesn't run at FP16 at all because they don't support it. You can emulate FP16 in FP32 math but since you'll be doing this manually you'll likely be slower than just using FP32, yes.

GCN3 introduced native FP16 support and packing of FP16 registers. GCN4 didn't change anything here I think.

Got to add that compilers will simply strip half precision hints from code targeted at platforms which don't support FP16 - and that's all there is to run the same code on OG PS4/Xbox One/Scorpio. The really tricky part is to find proper places to use partial precision in your high level shader code.
That clear up the things... thanks.

BTW I found that in Beyond3D.

sebbbi said:
Sometimes it requires more work to get lower precision calculations to work (with zero image quality degradation), but so far I haven't encountered big problems in fitting my pixel shader code to FP16 (including lighting code). Console developers have a lot of FP16 pixel shader experience because of PS3. Basically all PS3 pixel shader code was running on FP16.

It is still is very important to pack the data in memory as tightly as possible as there is never enough bandwidth to lose. For example 16 bit (model space) vertex coordinates are still commonly used, the material textures are still dxt compressed (barely 8 bit quality) and the new HDR texture formats (BC6H) commonly used in cube maps have significantly less precision than a 16 bit float. All of these can be processed by 16 bit ALUs in pixel shader with no major issues. The end result will still be eventually stored to 8 bit per channel back buffer and displayed.

Could you give us some examples of operations done in pixel shaders that require higher than 16 bit float processing?

EDIT:
One example where 16 bit float processing is not enough: Exponential variance shadow mapping (EVSM) needs both 32 bit storage (32 bit float textures + 32 bit float filtering) and 32 bit float ALU processing.

However EVSM is not yet universally possible on mobile platforms right now, as there's no standard support for 32 bit float filtering in mobile devices (OpenGL ES 3.0 just recently added support for 16 bit float filtering, 32 bit float filtering is not yet present). Obviously GPU manufacturers can have OpenGL ES extensions to add FP32 filtering support if their GPU supports it (as most GPUs should as this has been a required feature in DirectX since 10.0).

https://forum.beyond3d.com/posts/1805847/
 
This is a great thread. Awesome contributions from those who provide less complex examples, I have some programming knowledge but haven't specified 16 bit floats this way but it still makes perfect sense to me now. I'd guess that FP16 is best used where you can't see visual artifacts from the lower precision, where fast calculations needs to be done, and where you know that you won't end up with larger values.

In short. Perfectly okay for certain scenarios but not so okay for others. I guess it'll be the final tweaking in serious optimization.

Could we assume that FP16 is already used a lot among Sonys 1st party teams knowing their results? But maybe not by the third party devs? (Unless we're talking about Nixxes and the likes who'll try to beat the original devs whenever possible.)
 
Could we assume that FP16 is already used a lot among Sonys 1st party teams knowng their results? But maybe not by the third party devs? Unless we're talking about nixxes and the likes who'll try to beat the original devs whenever possible.
I don't believe it is already used because PS4 emulates FP16 over FP32 that is probably slower than just run FP32.

From the post I found in Beyond3D... PS3 used FP16 for pixel shader.
 
I don't believe it is already used because PS4 emulates FP16 over FP32 that is probably slower than just run FP32.

From the post I found in Beyond3D... PS3 used FP16 for pixel shader.
Okay but maybe for the Pro exclusive features not affecting the actual game code, maybe for downsampling, checkerboarding, 4K video streaming?
 
Okay but maybe for the Pro exclusive features not affecting the actual game code, maybe for downsampling, checkerboarding, 4K video streaming?
That is possible but I think it will be more used in the future... Pro is only 5 months old and devs didn't have much time to work in these type of optimizations.

I believe we will get a ton of info and demos next month with next nVidia GPUs and AMD Vega.
 
I think it wouldn't look as bad for textures in the general case as the gradient example. No reason you can't use this for geometry, right? I can imagine some models that would pass with 16bit float representation for vertex coordinates
 
Y'all should just become computer scientists, because at some point it's easier to spend a few years in school than read 10,000 forum posts and only understand 1/1000th as much.

Like the first post said, not a matter of smarts. Matter of knowledge. But you can appreciate this all a lot more if you learn in a complete / robust way (rather than bits and pieces).
 
Realistically speaking can we only expect up to a 50% performance boost from a combination of fp32 and fp16(mixed precision)? I'm certain nearly all the third party ports in the switch(at least Snake Pass, I am Setsuna, and Legocity Undercover) are using it as I don't think Switch could pull close performances to ps4 and xbone at 400GFLOPs vs mixed precision at 600GFLOPs which would make the xbone/ps4 2-3x more power performance instead of 3-4.5x without mixed precision. I.e. Correct me if I'm wrong please.

Though it also begs the question of how X1's newer architecture and tools bridges the power gap on the switch vs PS4 /xbone's older AMD architecture. This alone is a discussion worthy of its own topic.
 
Realistically speaking can we only expect up to a 50% performance boost from a combination of fp32 and fp16(mixed precision)? I'm certain nearly all the third party ports in the switch(at least Snake Pass, I am Setsuna, and Legocity Undercover) are using it as I don't think Switch could pull close performances to ps4 and xbone at 400GFLOPs vs mixed precision at 600GFLOPs which would make the xbone/ps4 2-3x more power performance instead of 3-4.5x without mixed precision. I.e. Correct me if I'm wrong please.

Though it also begs the question of how X1's newer architecture and tools bridges the power gap on the switch vs PS4 /xbone's older AMD architecture. This alone is a discussion worthy of its own topic.

There is nothing to suggest a 50% boost in performance for a gpu using mixed fp. You're not getting a gpu of any power to suddenly get 50% more peroformance. This is a means to optimize cases, very limited cases, using FP16 to help push to full utilization of a gpu.
 
I think it wouldn't look as bad for textures in the general case as the gradient example. No reason you can't use this for geometry, right? I can imagine some models that would pass with 16bit float representation for vertex coordinates
You can definitely store both model coordinates and normals in fp16. What you cannot do is transform coordinates out of model space via an fp16 transform, as you'd likely die for both range and precision. So you have to promote your fp16 coordinates to fp32 before undergoing transformations. That said, normals could be ok undergoing fp16 transforms.
 
Is there any information from developers how much of game engine can be coded in FP16 without visible lose of performance ?

I understand the technical but I wonder how much in percentage it could be used in modern game.

I wonder if Horizon isn't first one to use it - while simple ports to Pro usually had 1080p->1800 checkerboard Horizon jumps to 2160 checkerboard without performance degradation.
 
Something I saw on the slides but not mentioned in a post (or I missed it) is that fp16 also impacts throughput. Back when I was in school several years ago optimizing around memory bandwidth was very much a concern. I have no idea if that's a non-issue now (since caches are still pretty small, my guess is that it's still an issue) so it's not just a matter of calculation performance - there are wins across many steps (or losses if you mess up).
 
Is there any information from developers how much of game engine can be coded in FP16 without visible lose of performance ?

I understand the technical but I wonder how much in percentage it could be used in modern game.

I wonder if Horizon isn't first one to use it - while simple ports to Pro usually had 1080p->1800 checkerboard Horizon jumps to 2160 checkerboard without performance degradation.
~40% increase of pixel operations including those in fp32, really doubt fp16 can be hailed as the sole reason for this.
We will know more if we ever get a nice presentation on subject.

CPU part of game engine is coded the same, this doesn't affect CPU.
On GPU normal vertex sided like world to viewport transformations and such stay fp32.
Anything that needs high dynamic range stays the same. (Accumulating several lights etc.)

Things that are guaranteed to stay within a small numeric range are good candidates for fp16.
IE.
SSAO, you are within 0..1 as result, some games have used low precision data as source (8bit normal and low bit depth buffer and artifacts are small.).
Realistically speaking can we only expect up to a 50% performance boost from a combination of fp32 and fp16(mixed precision)?
For the whole game?
No, and it's just better to think it as an additional tool for developers, not magical % boost for every game that uses it.

Always think optimizations per shader, how long is the time in milliseconds before and after the optimization.
There are lot of possible bottlenecks in rendering from memory to not being able to run code parallel on GPU.
Something I saw on the slides but not mentioned in a post (or I missed it) is that fp16 also impacts throughput. Back when I was in school several years ago optimizing around memory bandwidth was very much a concern. I have no idea if that's a non-issue now (since caches are still pretty small, my guess is that it's still an issue) so it's not just a matter of calculation performance - there are wins across many steps (or losses if you mess up).
We have >10 flops per each byte, so I would guess bandwidth is still quite limited.
https://www.karlrupp.net/2013/06/cpu-gpu-and-mic-hardware-characteristics-over-time/
 
So is there any possible halfway house? An FP24? Running at 66% of time of FP32
In terms of current consoles that support FP16, no, there would be no gains to be had by running FP24. FP24 seems to be the odd duck in terms of precision, even though it offers HUGE scale and precision gains compared to FP16. But in terms of GPU's there is no native performance increase for it.

You have to think about GPU's as a bunch of small processors, each processor can handle (compute and store) in 32bits. That's easy, that's the way desktop GPU's have been designed for ages (with some minor exceptions which at the time were considered "cheating" at FP24).

Double Rate means that internal to each of those Processors, the architecture allows for native 16bit operations and storage, so a single processor can do two operations at the same time (where you could only do one 32bit one before, or waste potential precision for no performance gains by using FP16).

So 32bit = 16bit + 16bit (in terms of the GPU processor work)

24bit, is odd, because you'd only be left with 8 bits, float 8 would be pretty pointless, however INT8 makes sense in some cases (Nvidia Pascal supports quad rate INT8 btw). But GPU's don't support 24bit precision, with the remainder being able to be used for INT8.

That's not to say that it could be something that will happen in future silicon; running a 24bit float concurrently with an 8bit INT, some performance savings could be had. I would say that optimizing for this architecture advantage might be even less useful than the double rate FP16 though, FP16 is a simple precision change, all underlying use of those variables are the same, so it's pretty easy if the tradeoffs in precision arn't too obvious. changing types is more annoying, and in most cases not even reasonable.

Also, modern GPU's should be scheduling their calculations based on type anyways, so any INT8's in your existing code would be scheduled together with other INT8's already allowing for quad rate INT8's.
 
Does PS4 Pro also support INT8 like Vega? If so, do you get 4x performance compared to FP32?

I can see that being useful in limited cases (like AI code), which will certainly ease the load on Jaguar cores...
 
This is the second part of the fp16 precision-through-pictures analysis. This time it was carried on a device with native fp16 support (i.e. register storage and ALUs), eliminating the need for temporary texture storage. The native fp16 support allows us to examine a practical shading scenario without much effort. The device in question is an ARM Midgard T628 with native support for fp16, fp32 and fp64 arithmetics.

First, how does the power gradient reconstruction look on this device?
register storage fp16 said:

It looks very much like the fp16 test on Kepler. But if you overlay the two images, you'll notice that the Midgard picture has a slightly smaller black rectangle in the lower left - i.e. the extreme-low-values area. Why is that? Without actually checking, I'd venture to guess that Midgard supports denormalized fp16 values, while Kepler does not (in textures). Denormals narrow down the gap of non-representativeness around zero in every FP format.

Ok, on to the practical shading scenario. It comprises of a Blinn shader with tangent-space bump mapping. For maximum clarity we'll use that shader on a sphere with a homogenous, non-perturbed bump map (i.e. all normals in the map are (0, 0, 1)). We'll also re-use our bump map as an albedo map (programmer's art, etc).

entirely fp32 said:
entirely fp16 said:

It doesn't take much effort to notice the artifacts in the specular sport. As mentioned earlier, specular spots are produced via power functions, and those have the very useful property of turning precision candidate-issues into real such.

So, can we mend that via mixed precision? Let's see. We can examine the critical-precision path in our shader, and tweak that accordingly (the version of the shader shown below is already tweaked, so we'll just explain the tweaks).

mixed precision Blinn fragment shader said:

All fp32 quantities are marked in red. The grey-background area is something which would not normally be found in a production-quality shader pipeline. But our experimental shader is used on meshes without bi-tangent attributes, so the grey-area code computes the tangent-space transformation (TBN matrix) from partial derivatives at the fragment stage instead. In a production-quality shader this would be carried in the vertex shader via bi-tangent attributes in the vertex stream. So let's focus on the non-grey area of the shader.

For every computation (either an operator or a basic math function) which takes at least one fp32 operand, the computation will be carried in fp32. The texture sampling function is an exception - its texture coord argument does not affect the precision of the result, just how exactly the texture will be addressed. So what did we effectively tweak to fp32 in this critical path? We made sure the dot product of the Blinn "half-direction" vector and the bump-map normal is carried at fp32 precision. That's all.

The final result:

mixed precision said:
 
That needs a bit of context...

Before 2006 when shaders got unified in GPUs the games used to run with FP16/FP32 on nVidia or FP24 on ATIs... in case of nVidia the units can do 2x FP16 at the time of 1x FP32 and that give a good boost in performance for games that you can switch between both modes (FarCry, Half-Life 2, etc).

After 2006 the standard of unified shaders was created with FP32 only in mind... and so devs started to use fully FP32 and the GPUs focused in FP32 performance with the FP16 being slow compared with the FP32 unit (some GPUs runs FP16 at 1/8 of the speed of FP32).

But today AMD/nVidia are starting to look at the FP16 due performance gain... nVidia focused the mobiles GPUs like Tegra in FP16 running twice faster than FP32 and mobiles devs mainly develop with FP16 in mind. Now AMD is close to announce the Vega GPU that will compete with nVidia Pascal and the new feature is that FP16 run twice faster than FP32.

I guess AnandTech quote shows what exactly is happening with this move... today it will be not used but in the future games will take advantage of FP16 to get better performance of the GPUs.

Wasn't that the ill fated GeForce FX? I think the industry and the software best practices, as well as the feasibility to do so much processing with mixed FP32 and FP16 may lead to a rebirth of FP16... lovely how the industry goes in circles :).
 
Wasn't that the ill fated GeForce FX? I think the industry and the software best practices, as well as the feasibility to do so much processing with mixed FP32 and FP16 may lead to a rebirth of FP16... lovely how the industry goes in circles :).
Nothing new under the sun, they say.

I still have a geforceFX in a drawer of my desk here. I don't have a spare PCI slot in any of my machines, though, so I can't put that venerable specimen through the test.
 
Wasn't that the ill fated GeForce FX? I think the industry and the software best practices, as well as the feasibility to do so much processing with mixed FP32 and FP16 may lead to a rebirth of FP16... lovely how the industry goes in circles :).
Yeap... FX was pretty bad at FP32... that even generated some criticism to Valve forcing FP32 in Half-Life 2 as default (you need to go deep in config files to change tĂ´ FP16).

Radeon 9000 basically crushed the FX running everything with FP24.
 
I think it wouldn't look as bad for textures in the general case as the gradient example. No reason you can't use this for geometry, right? I can imagine some models that would pass with 16bit float representation for vertex coordinates

You can and this would be valid for memory bandwidth constrained situations. Keep in mind the API is going to convert the vertex data for the vertex shaders which, for anything other than mobile, are going to be float4 by default. Using fp16 for the scene vertex data is probably not going to work well. Back in the old days when we used to use 16-bit Z buffers you would see z-fighting a lot in distances. Better to just use float4 and not have to worry about it.
 
No one check mass effect andromeda? That is a new game running on one of the best engine out there with fp16 mode
We are talking strictly precision here, and as such, cases where we can compare screens produced via fp32 to the same or similar produced (partially) via fp16 are more interesting. Alas, DICE did not really discuss precision matters in their otherwise-excellent tech paper, so I'm not sure ME Andromeda really adds to the discussion.
 
So what did we effectively tweak to fp32 in this critical path? We made sure the dot product of the Blinn "half-direction" vector and the bump-map normal is carried at fp32 precision. That's all.
Is there any measurable difference in performance of the naive shader versus the mixed-precision version? Or versus the wholly FP32 implementation?

And thanks in general for this thread! It's very interesting, and clearly written.
 
No one check mass effect andromeda? That is a new game running on one of the best engine out there with fp16 mode

Andromeda's use of FP16 is limited to the checkerboard resolver on PS4 Pro and that's the only part that gets a 30% performance increase. It basically lowers it from 3ms per frame to 2ms per frame. The PS4 Pro supports double rate FP16 and the image involved is a RGBA16F frame buffer object (since you're resolving prior to tone mapping) so using FP16 is only natural in this case. It's entirely specific to the PS4 Pro's custom GCN core.
 
Is there any measurable difference in performance of the naive shader versus the mixed-precision version? Or versus the wholly FP32 implementation?
I haven't, as that shader is not really targeting optimal performance, e.g. a production shader would seldom need to compute its TBN transform per pixel. IOW, we'd need a somewhat better (read: more representative) case for a performance comparison, and that would've taken me more than the 30 minutes this test took me ; )

And thanks in general for this thread! It's very interesting, and clearly written.
I'm glad to be of service. /tips hat
 
Is there any measurable difference in performance of the naive shader versus the mixed-precision version? Or versus the wholly FP32 implementation?
Ok, on to the performance question.

I've rewritten the Blinn example above to be more like a production shader - the TBN transform is carried in the vertex shader, and the fragment shader looks like the quoted version sans the grey parts. I've also run the performance test on a very ALU-starved GPU; in contrast to the T628 MP6, this time the test was run on a T720 MP2 and much higher target resolution (1920x1200) with vsync locked. The fundamental difference between the cores of these two Midgards is that the T720 has one less ALU per core. Also, it's 6 cores for the T628 vs 2 cores for the T720. So the T720 is really at a disadvantage here, but mostly in ALU.

Here are the respective framerates at each stage of the T720 MP2 test:
  • original Blinn shader, fp32, with TBN transform in the fragment shader: 27 fps
  • "production version" -- re-factored shader with TBN transform in the vertex shader: 55 fps
  • "production version", fp16 in the fragment shader: 60 fps
Again, test was vsync'd through and through.
 
Top Bottom