dgrdsv
Member
That's actually really confusing to me, and I already had a loose understanding of the difference between the two.
My understanding is that FP32 is not "a number with 31 decimals" nor is FP16 "a number with 15 decimals".
Someone please correct me if I'm wrong, but my understanding was that - as the name implies - floating point numbers use a fixed number of bits and, basically, the decimal moves places in the numbers those bits represent.
So for very small numbers, you have a lot of precision after the decimal; e.g. 1.23456789 while larger numbers have less precision; e.g. 123456.789
Note: the above is not an accurate representation, only used to simplify the explanation for people that haven't dealt with this sort of thing before.
I just hope that's not another "simplification" which actually complicates matters.
From the horse's mouth so to speak:
It may affect image quality but if you won't be able to notice it it should be considered a valid performance optimization. On the other hand, when properly used, it should produce the exact same result as FP32 but noticeably faster (not even close to 2X though).Really, it's just that FP16 is trading precision for performance.
There are times when that additional precision is not going to be necessary, and should speed things up without affecting image quality.
Unfortunately I suspect that it's going to be used improperly at times, and that will hurt image quality.
HDR10 is using 10 bits per channel, HDR12 is using 12 bits, FP16 is more than enough for final image representation in both of these formats. Precision sensitive parts of code which may affect final image quality will still be done in FP32.And as we're moving to HDR, I would have thought that games would be starting to focus on using more precision, rather than less, but I'm sure there are places you can cut the precision without affecting the end result visually.
I do wonder how much of an overall speed-up it's going to be. It's certainly not going to double game performance for example.
We're supposedly only a couple of weeks from Volta unveil and I expect that there will be even more talk on mixed precision during this. FP16 and Int8 are highly usable in machine learning.