PSINext: People have come to associate high dynamic range lighting with on-chip hardware support for either FP16 or FP32 HDR rendering. How does Team Ninja implement NAO32, and can it be considered 'real' HDR?
Marco: The FP16 and FP32 rendering formats give a developer the opportunity to collect per pixel information (respectively 8 and 16 bytes per pixel); hence they easily enable us to render and to store an HDR image. Unfortunately, these framebuffer formats are inherently slow because they require more memory bandwidth and increased memory space: an FP16 720p image with 4X anti-aliasing requires about 30 MBytes of memory!
At the same time it's important to understand that it does not matter how we store our HDR images so long as we find a way to encode them without losing too much information.
The RGB color space is not very efficient at encoding HDR images, so after a bit of research we found another color space that is far more efficient at representing HDR images. Its name is CIE Luv, and it splits a color into 3 components: one is not normalized and represents how intense a color is (luminance), while the other 2 components are normalized between 0 and 1.
Gregory Ward, a pioneer of HDR imaging, exploited this color space many years ago to store HDR images in a file format he called LogLuv, so we built upon that work and we customized it to our purposes.
PSINext: So NAO32 is a means for you to preserve memory while at the same time retaining image quality, is that correct?
Marco: Correct. The main idea behind NAO32 is that we want to trade shading power to regain memory space and bandwidth (very precious resources on a console). So instead of encoding our HDR colors into a FP16 or FP32 frame buffer, we devised a scheme to use RSX pixel shading units to convert an RGB color in a CIE Luv color that only requires a common RGBA8 frame buffer (4 bytes per pixel, half the space of a FP16 pixel) to be fully stored.
The quality of this format is really outstanding. Even if it uses half the space/bandwidth of common HDR rendering solutions, it really makes no compromises at all in image quality.
There's no magic here: HDR rendering costs are shifted from memory to shaders, and so our shaders are a bit longer now (between 3 and 5 cycles). We believe it's a very good trade-off. Furthermore, it enables HDR rendering and multisample anti-aliasing on GPUs that do not natively support AA with floating point render targets such as FP16 and FP32.
We also developed a faster 3 bytes per pixel version called NAO24 (predictable, isn't it?) that supports a narrower dynamic range with less accuracy. And although the quality was quite decent in most cases, we decided against making any compromises, and so in the end we did not use it.
My final answer is totally positive: NAO32 can be considered a real HDR format.
I like Jojo. Forgive me.
