• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Kraken Vs ZLIB: 29% Smaller Game Sizes Losslessly, 297% Faster Decompression on PS5

Shmunter

Member
Overall both should have enough storage with future games. I still doubt that any off the shelf NVMe m.2 7GB/s will match PS5's internal custom 5.5GB/s SSD, even though that 7GB/s top tier NVMe m.2 SSD's are vastly better than Xbox cut-down (2x lanes instead of 4x) low budget (only 4 channel for around 16 chips!) SSD's. I think Sony should just release a 825GB extension, and they might do that but they avoided it now to avoid bad press.
PS5 needs cold storage. Sony’s glacial approach to updates is pissing me off already.
 

M1chl

Currently Gif and Meme Champion
Overall both should have enough storage with future games. I still doubt that any off the shelf NVMe m.2 7GB/s will match PS5's internal custom 5.5GB/s SSD, even though that 7GB/s top tier NVMe m.2 SSD's are vastly better than Xbox cut-down (2x lanes instead of 4x) low budget (only 4 channel for around 16 chips!) SSD's. I think Sony should just release a 825GB extension, and they might do that but they avoided it now to avoid bad press.
I don't really argue that, but it's also more to do how well the RTX i/o is going to be, as far as I know, both consoles are sporting dedicated hw i/o block, which even as a more budget options in XSX is more capable than classical CPU decompressor found in PC. Will see, RTX i/o is supposedly build on Direct Storage API, so those solutions are going to be interesting to watch, what the crazy power different is going to do.
 

Bo_Hazem

Banned
I don't really argue that, but it's also more to do how well the RTX i/o is going to be, as far as I know, both consoles are sporting dedicated hw i/o block, which even as a more budget options in XSX is more capable than classical CPU decompressor found in PC. Will see, RTX i/o is supposedly build on Direct Storage API, so those solutions are going to be interesting to watch, what the crazy power different is going to do.

I think both AMD and Nvidia cards have so much juice to spare to mimic console I/O decompressors. But AMD can go the extra mile and release crazy CPU's that can free the GPU from that as PS5's decompressor alone is as powerful as 9x Zen2 cores, with the I/O being around 11-12x Zen2 cores! That's like having entry level AMD threadripper CPU dedicated for the I/O alone.
 
Last edited:

M1chl

Currently Gif and Meme Champion
I think both AMD and Nvidia cars have so much juice to spare to mimic console I/O decompressors. But AMD can go the extra mile and release crazy CPU's that can free the GPU from that as PS5's decompressor alone is as powerful as 9x Zen2 cores, with the I/O being around 11-12x Zen2 cores! That's like having entry level AMD threadripper CPU dedicated for the I/O alone.
ASIC > CPU also probably needs like 1/20 of die size of those said CPU cores. Well I hope that we get some (even a demo-tech) game soon with what it can do, well will see how Ratchet and Clank will be, even though it's still early gen game.
 

Bo_Hazem

Banned
ASIC > CPU also probably needs like 1/20 of die size of those said CPU cores. Well I hope that we get some (even a demo-tech) game soon with what it can do, well will see how Ratchet and Clank will be, even though it's still early gen game.

R&C is still using traditional LOD's, but distant assets are crazy sharp that you might think they're simply polygon/atom streaming!
 

M1chl

Currently Gif and Meme Champion
R&C is still using traditional LOD's, but distant assets are crazy sharp that you might think they're simply polygon/atom streaming!
Wait "traditional LOD" as oppose to those UE5 nanites? Or is it something more which I missed?
 

Bo_Hazem

Banned
Wait "traditional LOD" as oppose to those UE5 nanites? Or is it something more which I missed?

Making 5-9 levels of details of the same asset is the traditional way. Sony Atom View / Unreal Engine 5 can use one original asset and stream polygons/atoms on the fly in a frame budget instead, which is insanely efficient and should make even old GPU's more powerful and good enough to make better graphics. It's in the OP.
 

M1chl

Currently Gif and Meme Champion
Making 5-9 levels of details of the same asset is the traditional way. Sony Atom View / Unreal Engine 5 can use one original asset and stream polygons/atoms on the fly in a frame budget instead, which is insanely efficient and should make even old GPU's more powerful and good enough to make better graphics. It's in the OP.
Oh
I rarely reads OP with multiple pages
 
Because those old games and some more other games aren't coded to take full advantage of either. 1st party games so far are doing below 1sec up to 3-4sec max, and they look much more better than those games.



Should be good enough, but I don't think it'll compete well if it's still using ZLIB on Xbox, as devs will rather save money than license Oodle Kraken individually for their game on Xbox, while it's free + Oodle Texture on PS5/4.
It may end in Sony's favor and push microsoft to follow what sony did to compete , just like what they did with the one x.
 

quest

Not Banned from OT
It may end in Sony's favor and push microsoft to follow what sony did to compete , just like what they did with the one x.
Microsoft has different goals they need to release games day and date on 2 platforms. They can't be using any expensive proprietary technology that won't be adopted on the second platform. Microsoft took an approach that works great to bring big improvements to 2 platforms with widespread adoption. Sony has a bad ass io but works in a closed system.
 
Microsoft has different goals they need to release games day and date on 2 platforms. They can't be using any expensive proprietary technology that won't be adopted on the second platform. Microsoft took an approach that works great to bring big improvements to 2 platforms with widespread adoption. Sony has a bad ass io but works in a closed system.
But if they look that different there should be a point where it's unacceptable for them to not take any action, there's always risks and beds been taken for the future tech ,it's not always 100 percent clear for any of them.
 

Bo_Hazem

Banned
But if they look that different there should be a point where it's unacceptable for them to not take any action, there's always risks and beds been taken for the future tech ,it's not always 100 percent clear for any of them.

Also the Oodle Kraken can work good even with ZLIB decompressors like in Xbox consoles and PS4.
 
Last edited:

Bo_Hazem

Banned
Texture streaming is faster on PS5 than Xbox Series X and Xbox Series S http://bit.ly/3rzcVut Note that these screenshots are right after loading and these are not representative of how the game generally looks on the Xbox consoles.

PS5

ps5-streaming-3.png


Xbox Series X

seriesx-streaming-3.png


Xbox Series S

seriess-streaming-3.png


Struggling with low res textures in a BC game? (Mortal Shell)
 
Last edited:
Texture streaming is faster on PS5 than Xbox Series X and Xbox Series S http://bit.ly/3rzcVut Note that these screenshots are right after loading and these are not representative of how the game generally looks on the Xbox consoles.

PS5

ps5-streaming-3.png


Xbox Series X

seriesx-streaming-3.png


Xbox Series S

seriess-streaming-3.png


Struggling with low res textures in a BC game? (Mortal Shell)


That is rather interesting... They could probably extend the initial loading screen time to allow these assets to filter in behind the scenes before letting the player loose, seems like an easy fix, but who knows.
 

Bo_Hazem

Banned
That is rather interesting... They could probably extend the initial loading screen time to allow these assets to filter in behind the scenes before letting the player loose, seems like an easy fix, but who knows.

It could be (sorry for my late response). Being smaller is one thing, but streaming 297% faster + PS5's I/O speed makes it even more critical in future exclusives and how they're designed. R&C using extremely high assets while still relatively small is pretty impressive as well.





Also with PS5 updates don't accumulate like in PS4, so updates can be big but the game can stay nearly the same:

 
It could be (sorry for my late response). Being smaller is one thing, but streaming 297% faster + PS5's I/O speed makes it even more critical in future exclusives and how they're designed. R&C using extremely high assets while still relatively small is pretty impressive as well.





Also with PS5 updates don't accumulate like in PS4, so updates can be big but the game can stay nearly the same:



Agreed! I seriously cannot wait until UE5 (Subject change lol) makes into the mainstream especially if we're talking nanite and lumen. That demo is absolutely impressive (Streaming/geometry fidelity). Seriously, the only thing Microsoft did this generation is try to check the raw compute crown checkbox. I am still slapping myself when I play PS5 titles that have little to no load times.
 

Bo_Hazem

Banned
Agreed! I seriously cannot wait until UE5 (Subject change lol) makes into the mainstream especially if we're talking nanite and lumen. That demo is absolutely impressive (Streaming/geometry fidelity). Seriously, the only thing Microsoft did this generation is try to check the raw compute crown checkbox. I am still slapping myself when I play PS5 titles that have little to no load times.

That's literally scratching the surface. R&C is still using LOD's, so there is wasteful size, unlike HDD wasteful sizes though that have duplicates, but that could be cut down to 1 asset and stream polygons on the fly by using frame budget instead of proximity with LOD's (5-7 levels of the same asset). This means no pop ins and draw distance can be near infinite because you only dealing with what fills the frame/resolution.
 

Kerotan

Member
Besides wz I'm very happy with PS5 game sizes. But then again wz is a ps4 game.

So happy I can easily see myself waiting 2 years for an internal ssd. Maybe longer. 500mb Internet too games download in no time.
 

Bo_Hazem

Banned
Besides wz I'm very happy with PS5 game sizes. But then again wz is a ps4 game.

So happy I can easily see myself waiting 2 years for an internal ssd. Maybe longer. 500mb Internet too games download in no time.

I really think with proper internet you don't need external storage, but it's a welcome addon.
 

Bo_Hazem

Banned
Thats whats bad here in the US so many people have poor internet and many do not even have access to any kind of wired internet at all being forced to rely on satellite internet in rural areas.

I think Sony should release NVMe SSD built like their internal SSD (6 priority levels vs 2, 12-channel per 12 chips 1:1 instead of 1:2 on high-end NVMe SSD's) and help other partners because it can be used on PC as well.
 

Panajev2001a

GAF's Pleasant Genius
I think Sony should release NVMe SSD built like their internal SSD (6 priority levels vs 2, 12-channel per 12 chips 1:1 instead of 1:2 on high-end NVMe SSD's) and help other partners because it can be used on PC as well.
They would need custom drivers and extend the existing NVMe specs which means that MS would also need to support them in DX (Direct Storage) for this feature to be properly used (extra priority levels).
 

Bo_Hazem

Banned
They would need custom drivers and extend the existing NVMe specs which means that MS would also need to support them in DX (Direct Storage) for this feature to be properly used (extra priority levels).

Why wouldn't MS support it? Do they want people to go Linux as it's starting to welcome some games AFAIK? It could be supported by Apple as well if they wanna act like a smart ass.

Data Caps exist.

Indeed they do and already thanks to all movies streaming + wfh hitting them is quite easy.

With proper internet they don't.

Honestly, not sure what happened in the US. Back in 2002-2004 I remember both AOL Dial-up then COX cable had no data caps. Here we have caps for 4-5G, but ADSL/Fiber with all packages are unlimited data, although they can be much more expensive than in the US. I'm paying 40 OMR ($104) monthly for 200Mbps down/50Mbps up + 4K streaming service reciever known as Jawwy TV:


If there is an option for unlimited I'll always go for it, you can redownload the game with 200Mbps in around an average of 10-35min while playing something until it's finished, that's why I would rather delete the game I'm finished with and that's not getting any new DLC's. But at least support a faster cold storage with NVMe that can transfer games much faster in/out would help people with caps or bad internet in a 7GB/s speed instead of 512MB/s with a SATA3 SSD external or 1.2GB/s for external NVMe (10Gbps USB = 1.25GB/s).
 
Last edited:
Data Caps exist.
Australia use to have really bad internet and data caps. Now we fairly good internet (up to 1gbs on some plans for residential at $116usd) and nearly all plans are unlimited. Sheesh you can get mobile 5g broadband for $79 unlimited. How do we an island so far away from everyone get no data caps? Having a quick look and Canada seems to be terrible
 
Last edited:
Australia use to have really bad internet and data caps. Now we fairly good internet (up to 1gbs on some plans for residential at $116usd) and nearly all plans are unlimited. Sheesh you can get mobile 5g broadband for $79 unlimited. How do we an island so far away from everyone get no data caps? Having a quick look and Canada seems to be terrible
I live in the second smallest province in Canada. 109$cdn 1.5gbs up and down no cap.
 

bender

What time is it?
Australia use to have really bad internet and data caps. Now we fairly good internet (up to 1gbs on some plans for residential at $116usd) and nearly all plans are unlimited. Sheesh you can get mobile 5g broadband for $79 unlimited. How do we an island so far away from everyone get no data caps? Having a quick look and Canada seems to be terrible
tenor.gif
 
Australia use to have really bad internet and data caps. Now we fairly good internet (up to 1gbs on some plans for residential at $116usd) and nearly all plans are unlimited. Sheesh you can get mobile 5g broadband for $79 unlimited. How do we an island so far away from everyone get no data caps? Having a quick look and Canada seems to be terrible
In Germany the Situation despite the defitism spread by leftists is actually realy good.
I moved 5 Times in the Last 20years and had always Proper inet speeds available to me..
I also some sort of early adopter for cable modems.
Whenever they offered me higher bandwidth I took it.
Brought me to now 1000mbits /s (~120MB/s) for a mere 50€/month.
Uncapped of course...
I believe capped inet contracts died out in 1999 or so here. ;)
 

Haggard

Banned
I think both AMD and Nvidia cards have so much juice to spare to mimic console I/O decompressors. But AMD can go the extra mile and release crazy CPU's that can free the GPU from that as PS5's decompressor alone is as powerful as 9x Zen2 cores, with the I/O being around 11-12x Zen2 cores! That's like having entry level AMD threadripper CPU dedicated for the I/O alone.

The PS5 has specialized silicone for a very special use case requiring very specific data formats.
This "worth xy cpu cores" is complete nonsense as long as these very specific circumstances aren't met.
This marketing hogwash always makes it sound as if there was an extra PC bundled into the consoles.....
 

Bo_Hazem

Banned
The PS5 has specialized silicone for a very special use case requiring very specific data formats.
This "worth xy cpu cores" is complete nonsense as long as these very specific circumstances aren't met.
This marketing hogwash always makes it sound as if there was an extra PC bundled into the consoles.....

Actually for now Nvidia mimicking i/o, direct storage in general have to prove themselves, but reality is that is an impossible task to overcome with all the latency in the aging PC setup.

Man. People evangelizing consoles is tough to read. This thread is like it was posted on the Sony version of The_Donald.

Your post is really worthless and baseless, like the one above.

Wow you, omani, should be very rich, i'm in a poor country and paying just ~8$ for 1Gbit connection 😏

Far from being rich, but $8 for 1Gbps!

fuck my life fml GIF
 
Last edited:
The PS5 has specialized silicone for a very special use case requiring very specific data formats.
This "worth xy cpu cores" is complete nonsense as long as these very specific circumstances aren't met.
This marketing hogwash always makes it sound as if there was an extra PC bundled into the consoles.....

Not really the decompressor in these consoles are there for a very specific purpose. It's not like having another CPU inside of them. The reason they explain their strengths in terms of CPU cores is because it's easier to understand how capable the decompressors are. It doesn't mean you can use them to do other tasks besides decompressing data.

When compared to PCs Bo_Hazem Bo_Hazem and Tim Sweeney are correct in that consoles are capable of certain customizations that can't be used in PCs. If PCs were not a mish mash of parts from different manufacturers it would be a different story. It doesn't mean PCs cant surpass the I/O performance of these systems it's just PC can't use the same methods.

Systems integration and whole-system performance. Bringing in data from high-bandwidth storage into video memory in its native format with hardware decompression is very efficient. The software and hardware stack go to great lengths to minimize latency and maximize the bandwidth that's actually accessible by games.

Those PC numbers are theoretical and are from drive into kernel memory. From there, it's a slow and circuitous journey through software decompression to GPU driver swizzling into video memory where you can eventually use it. The PS5 path for this is several times more efficient. And then there's latency.

On PC, there's a lot of layering and overhead. Then you have the issue of getting compressed textures into video memory requires reading into RAM, software decompressing, then calling into a GPU driver to transfer and swizzle them, with numerous kernel transitions throughout.

Intel's work on non-volatile NVDIMMs is very exciting and may get PC data transfer on a better track over the coming years.
 
Last edited:

Bo_Hazem

Banned
Not really the decompressor in these consoles are there for a very specific purpose. It's not like having another CPU inside of them. The reason they explain their strengths in terms of CPU cores is because it's easier to understand how capable the decompressors are. It doesn't mean you can use them to do other tasks besides decompressing data.

When compared to PCs Bo_Hazem Bo_Hazem and Tim Sweeney are correct in that consoles are capable of certain customizations that can't be used in PCs. If PCs were not a mish mash of parts from different manufacturers it would be a different story. It doesn't mean PCs cant surpass the I/O performance of these systems it's just PC can't use the same methods.

Thanks for the addition, although I won't combine them as "consoles" because there are critical differences between PS5 and XSX, and I'm 100% that nVidia's 14GB/s I/O will surpass XSX easily:

XSX:

- Low-tier NVMe SSD, 4-channel (vs 8-channel high end PC and 12-channel PS5).
- DRAM-less (because of that, main CPU must do the seek vs Samsung DRAM on PS5 and SRAM on the I/O unit).
- 2x PCIe 4.0 lanes instead of 4x on PC and PS5.
- CPU on XSX assists its ZLIB decompressor vs Kraken decompressor on PS5 is 100% independent.
- 2 layers of motherboards on XSX adds more latency vs 1 motherboard on PS5.
- Lack of GPU cache scrubbers = prone to stalls like PC and less bandwidth efficiency.

We would've seen the UE5 demo running on both consoles if XSX was close enough, but it's a slightly modified PC with shared Windows OS (even though if lighter, not ideal for 1 system but great for easy PC=Console ports).
 
Last edited:

Dozer831

Neo Member
Kraken VS ZLIB:

According to the official graph by RAD Game Tools, Kraken has 29% higher compression ratio over ZLIB (used in PS4 and other platforms), and does it 3-5x faster (doesn't really concern us as gamers, only good for devs). Both Kraken and ZLIB are lossless compressions, meaning data is kept as it is, like textures, audio, game files, etc. Also another advantage is being 297% (~4x times) faster in decompression, which is very critical for data streaming at least.

oodle-typical-vbar.png


Oodle Texture:

Textures in games take a big chunk of the total size. To amplify compression even further, Oodle Texture go 5-15% "near" lossless compression:

Lossless Transform for BC7: Oodle Texture also includes a lossless transform for BC7 blocks called "BC7Prep" that makes them more compressible. BC7Prep takes BC7 blocks that are often very difficult to compress and rearranges their bits, yielding 5-15% smaller files after subsequent compression. BC7Prep does require runtime reversal of the transform, which can be done on the GPU. BC7Prep can be used on existing BC7 encoded blocks, or for additional savings can be used with Oodle Texture RDO in near lossless mode. This allows significant size reduction on textures where maximum quality is necessary.

oodle-texture-bc7prep-ratios.png


Pushing compression with a slight loss, you can go up to 50% smaller textures files:

Oodle Texture Rate Distortion Optimization (RDO), sometimes known as "super compression", lets you encode BCN textures with your choice of size-quality tradeoff. Oodle Texture RDO searches the space of possible ways to convert your source texture into BCN, finding encodings that are both high visual quality and smaller after compression. RDO can often find near lossless encodings that save 10% in compressed size, and with only small visual difference can save 20-50%.

Oodle Texture RDO provides fine control with the lambda parameter, ranging from the same quality as non-RDO BCN encoding, to gradually lower quality, with no sudden increase in distortion, or unexpected bad results on some textures. Oodle Texture RDO is predictable and consistent across your whole data set; you can usually use the same lambda level on most of your textures with no manual inspection and tuning. Visit Oodle Texture RDO examples to see for yourself.

RDO encoding just produces BCN block data, which can be put directly into textures, and stored in hardware tiled order. Oodle Texture RDO does not make a custom output format and therefore requires no runtime unpacking.


Kraken Hardware Decompressor:

That said, Kraken hardware decompressor (equivalent of 9x Zen2 cores) on PS5 should work best with Oodle Texture, but Oodle Texture can work with any lossless decompressor, like ZLIB HW decompressor on Xbox Series X|S (equivalent of 3x Zen2 cores), and PS4. Xbox Series X|S already have a great solution for texture compression known as BCpack.

Oodle Texture RDO optimizes for the compressed size of the texture after packing with any general purpose lossless data compressor. This means it works great with hardware decompression. On consoles with hardware decompression, RDO textures can be loaded directly to GPU memory, Oodle Texture doesn't require any runtime unpacking or CPU interaction at all. Of course it works great with Kraken and all the compressors in Oodle Data Compression, but you don't need Oodle Data to use Oodle Texture, they are independent.

Sources:



PS5 vs Xbox Series X

As Xbox Series X|S have around similar texture reduction, we can't say Oodle Texture is better than BCpack nor vice versa. But Sony has licensed Oodle Kraken + Texture to all PS4/PS5 developers and it still has 29% size reduction advantage over ZLIB on PS4 and Xbox Series X|S, mated with Kraken hardware decompressor. We also know that Xbox Series S will have gimped versions of the Series X textures, so will put it aside from the comparison. Assuming the textures/assets are the same on both consoles:

meta-chart.jpg


But, with the 297% (~4x times) faster decompression, which should be even better with Kraken HW decompressor, PS5 might leverage the extra space to use higher quality assets.

It doesn't stop right there though, with PS5 unprecedented SSD and I/O, it could eliminate the use of LOD's (Levels of Details), which can be 5-7 images in different resolutions/qualities. Sony Atom View has brought this technique in 2017, using only one high quality asset and streaming polygons as you get closer, resulting in much higher quality textures without wasting size on duplicates (5-7 versions of the same asset) in the old fashion LOD system. Atom View works as a plugin on UE and Unity engines, and probably with other Sony WWS proprietary engines:





Sony has helped Epic Games achieve similar tech known as "Nanite" on their upcoming Unreal Engine 5, using hundreds of billions of polygons of uncompressed 8K Hollywood-level assets with up to 16K shadows, all crunched losslessly to around 20 million polygons per frame for this 4.2 million pixel (1440p) gameplay demo. It only needed a 0.7GB pool of RAM for streaming:

(Timestamped)





The I/O throughput of 22GB/s makes the whole 825GB SSD like a DDR4 RAM! Which is insane:

DDR4 data rates: (single channel)

DDR4 2133: 17 GB / s
DDR4 2400: 19.2 GB / s

DDR4 2666: 21.3 GB / s
DDR4 3200: 25.6 GB / s


TL;DR

PS5 should benefit from this headroom advantage to use higher quality assets than other platforms, or be more efficient with its storage for the same assets quality but smaller game sizes. All this while still being able to eliminate LOD's and use that extra space to have a bump on assets quality for the same smaller game size overall!

Thank you for reading.

EDIT: Thanks to @GreyHand23 for the addition:



PS:

Recent results: (XSX|S version is 68% larger "42GB" than PS5 version "25GB")


An impressive 71% file reduction from 12.910GB to 3.753GB! Credit to Anchovie123 Anchovie123

MwvuFZJ.jpg

I think the difference in file size is because the XSX is using the same file size as the PC for installs
 

Bo_Hazem

Banned
I think the difference in file size is because the XSX is using the same file size as the PC for installs

Devs with licensed Kraken compressor and Oodle Texture will have better compression on Xbox than what's offered right now (open-source ZLIB and BCPack) and will decompress 3x faster as well. Could be included with any game developed on UE4/5 as Epic Games bought RAD Tools.
 
Last edited:

Utherellus

Member
Both of consoles will handle compression and streaming brilliantly. Both of them went different routes to accomplish same goal.

Though I'm sad that MS isn't advertising and most importantly not accelerating the release of Sampler Feedback Streaming which is groundbreaking feature by further giving 2-3x multiplier beyond 4.8GB/s I/O bandwidth.


32592128784126222014.jpg



27427512720993572258.jpg


80964816544782384443.png
 
Both of consoles will handle compression and streaming brilliantly. Both of them went different routes to accomplish same goal.

Though I'm sad that MS isn't advertising and most importantly not accelerating the release of Sampler Feedback Streaming which is groundbreaking feature by further giving 2-3x multiplier beyond 4.8GB/s I/O bandwidth.


32592128784126222014.jpg



27427512720993572258.jpg


80964816544782384443.png

I wouldn't say the end results will be the same. These systems are not clones of each other and each have their own advantages over the other.
 
Both of consoles will handle compression and streaming brilliantly. Both of them went different routes to accomplish same goal.

Though I'm sad that MS isn't advertising and most importantly not accelerating the release of Sampler Feedback Streaming which is groundbreaking feature by further giving 2-3x multiplier beyond 4.8GB/s I/O bandwidth.

Yeah, around 6 is max. It was discussed here to the death. There is no +10 GB/s on XSX


 
Last edited:
Top Bottom