Support NeoGAF

[KojiKnight]

Why would transparent compression have significant effect on SSD wear?

In fact, it should be the opposite. Less space used means less accessing when reading or writing meaning less wear and tear since all of the decompression is on the fly and happens in memory.

 

[Xcell Miguel]

In fact, it should be the opposite. Less space used means less accessing when reading or writing meaning less wear and tear since all of the decompression is on the fly and happens in memory.

Not when games get patched, as you might need to unpack the files to get properly patched (Ghost Recon Wildlands for example, gets broken because its files can't be properly patched while packed), or it would get patched and then need to be packed again.

I'll advise to only use this on "old" games you want to archive but still play from time to time, and that may not be patched anymore.

 

[KojiKnight]

Not when games get patched, as you might need to unpack the files to get properly patched (Ghost Recon Wildlands for example, gets broken because its files can't be properly patched while packed), or it would get patched and then need to be packed again.

I'll advise to only use this on "old" games you want to archive but still play from time to time, and that may not be patched anymore.

This should still be happening on the fly in memory. Ghostlands has to be doing something weird, like skirting the OS to write the files directly to the hard drive.

That said, most game 'patches' don't actually patch files, but replace them with new ones. Far less likely for something to go wrong and shouldn't cause any issues.

 

[KainXVIII]

Not bad result for FFX HD

But Total Commander or Windows Explorer still shows same amount on free space for HDD (i think), so, in practical way, what did i gain after compressing and how to see it?

 

[snail_maze]

Not bad result for FFX HD

But Total Commander or Windows Explorer still shows same amount on free space for HDD (i think), so, in practical way, what did i gain after compressing and how to see it?

i was just going to ask the exact same question, haha
So my "size on disk" is reduced, but the free space on my drive is i think still the same

 

[Black_Stride]

Not bad result for FFX HD

But Total Commander or Windows Explorer still shows same amount on free space for HDD (i think), so, in practical way, what did i gain after compressing and how to see it?

Woah thats insane i thought i got good results with Spirit of Sanada going from 24 to 7GB.

But thats absolute madness....why didnt i know about XPRESS compression till now?

 

[KojiKnight]

Not bad result for FFX HD

But Total Commander or Windows Explorer still shows same amount on free space for HDD (i think), so, in practical way, what did i gain after compressing and how to see it?

i was just going to ask the exact same question, haha
So my "size on disk" is reduced, but the free space on my drive is i think still the same

Did you guys remember to hit "refresh" on the folder that shows your hard drive space? Cause I just tried it to make sure and it's definitely freeing up the space.

 

[pooptest]

Do you think this will have any effect on wear and tear of an SSD? I might compress my Blu-ray rips with this as well

I understand write levels for a business having some sort of concern, but for home users this is the last thing I’d generally worry about with an SSD.
The warranty alone for a Samsung 850 Pro 256GB is 150TBW, which is about 41GB/day for 10 years. And the 850 Pro has been tested at upwards of 9.1PB before finally dying. The larger the disk, the more endurance.

Only thing I’d be concerned about is getting an SSD that uses MLC instead of TLC. I don’t know if they make SLC consumer drives, but that’d be even better.


Edit: quoted wrong person (fixed).

 

[snail_maze]

Did you guys remember to hit "refresh" on the folder that shows your hard drive space? Cause I just tried it to make sure and it's definitely freeing up the space.

Yes, also restarted my PC

 

[KainXVIII]

Yes, also restarted my PC

Same, no difference.

 

[Xcell Miguel]

This should still be happening on the fly in memory. Ghostlands has to be doing something weird, like skirting the OS to write the files directly to the hard drive.

That said, most game 'patches' don't actually patch files, but replace them with new ones. Far less likely for something to go wrong and shouldn't cause any issues.

Not really, for example this game uses large 2 GB to 20 GB files, they can all be patched with delta updates, without needing to redownload the whole files on Uplay.

Same for DiRT 4, the game is 35 GB, and it got many 500 MB patch near launch, but because of the way it's made, Steam had to copy 30 GB of the game's data as the "source" to then patching in the 500 MB downloaded because it patched exactly some part of many big files with only 500 MB of delta.

So if it's packed with such compression, delta patching can be a problem as the data is not stored the way the patching tool expects it.

 

[FunkyDealer]

Apparently the program currently has issues if your system language is not set to English. So proceed with caution, or wait for a fix.

Yeah, my bad. Was a bit tired when making the post.

No worries, I managed to make a mistake in my short little reply as well. But I figured it was worth pointing out your mistake, as it could give people some false hopes.

 

[cyress8]

Will use this when I get home later. Compress the games I play once in a blue moon.

 

[patientx]

Tried it on my laptop with offline steam ,

Elex, from ~26GB to ~6GB. I think this tool exposes developers who didn't compress the files at all.. And yes, the game runs well after compression.

Project CARS on the other hand just got ~1GB smaller from 46 to 45 . So not worth for it.

 

[snail_maze]

Tried it on my laptop with offline steam ,

Elex, from ~26GB to ~6GB. I think this tool exposes developers who didn't compress the files at all.. And yes, the game runs well after compression.

Project CARS on the other hand just got ~1GB smaller from 46 to 45 . So not worth for it.

And the free space of your drive in windows explorer increased by that amount too?

 

[MUnited83]

Tried it on my laptop with offline steam ,

Elex, from ~26GB to ~6GB. I think this tool exposes developers who didn't compress the files at all.. And yes, the game runs well after compression.

Project CARS on the other hand just got ~1GB smaller from 46 to 45 . So not worth for it.

It doesn't "expose" anything, you still need CPU overhead, which means the games would just have higher minimum requirements and being able to be played by less people. No thanks.

 

[patientx]

And the free space of your drive in windows explorer increased by that amount too?

Yes, with Elex it did.

 

[patientx]

It doesn't "expose" anything, you still need CPU overhead, which means the games would just have higher minimum requirements and being able to be played by less people. No thanks.

Well, I only have 240GB SSD for games and my main windows hdd is 128GB , so every GB counts for me. 20 GB example here is great for me.

 

[snail_maze]

Yes, with Elex it did.

Ok, thanks.
Going to have to play around with this some more later

 

[Kayant]

Thanks for the heads up I have been wanting to compress some folders but 7zip takes too long this seems to do a have decent compression ratio and is fast.

Might try it on some games later.

 

[KainXVIII]

Compressed Rime on my SSD (nearly 60% saved), but free space still shows unchanged 🤐

 

[Knurek]

Compressed Rime on my SSD (nearly 60% saved), but free space still shows unchanged 🤐

Are you using the GUI tool?
If so, try just running this command from the game folder:

compact /C /S /EXE:XPRESS16K

 

[patientx]

Compressed Rime on my SSD (nearly 60% saved), but free space still shows unchanged 🤐

The directory itself shows the same number , check the HDD free space on windows explorer window. For testing I open the explorer window with detailed view and after compression free space changes. And maybe, w10 must be up to date ? I installed latest fall update today maybe that is what shows it correctly ? I am not sure about it though. Just a thought.

 

[KainXVIII]

Well, it seems free space is a little changed, but not much (and not changed for FFXHD on another HDD, where 37gb compressed to only ~4gb &#128512 And yes, Fall update installed (and fucked up my sound card drivers, thanks MS, but that's another story)

Free space before

CGui report

Folder info

Free space after

 

[patientx]

My last post for this,

Look at size and size on disk differences. Siege also compressed great. "Temel" is various drivers and programs I keep for installs.

 

[Coonce]

My last post for this,

Look at size and size on disk differences. Siege also compressed great. "Temel" is various drivers and programs I keep for installs.

That siege compression is great, have you tested the map loading times in lobbies yet? after installing the ultra HD pack for siege, my load times went way up but I haven't compressed it yet.

 

[wwm0nkey]

Has anyone tried Titan Fall 2

 

[Coonce]

Has anyone tried Titan Fall 2

I did, saved 23 GB.

game runs just fine!

 

[Sciz]

I'm not saying Compact doesn't compress things. I'm just saying whatever Compact does, SteamCDN does better. Mainly due to the fact that it's not a runtime compression.

(Don't have any of the titles you mention, but Cuphead is a Unity title, and should behave exactly as the two games I mentioned).

Cuphead, IIRC, was a 1.9 GB download. Couldn't believe it when it installed as 10.9 GB.

 

[Majukun]

nevermind

 

[Paragon]

Not bad result for FFX HD https://i.imgur.com/sTorxF8.png
But Total Commander or Windows Explorer still shows same amount on free space for HDD (i think), so, in practical way, what did i gain after compressing and how to see it?

Total Commander must be reporting "Size" rather than "Size on Disk". There's probably an option to change that somewhere.

Does this make the compressed files unreadable from other oses? like osx?

It's only supported on Windows 10.

Cuphead seems to handle compression really well, went from 11GB to 2.75GB. I wonder if this could be used to deliver games through Steam and the Win10 Store. If they could be downloaded heavily compressed as this and then unpacked on the HDD after, it would be great for people with datacaps.

Hollow Knight is similar. 9GB down to 1.65GB using LZX.
Game doesn't seem to be affected by this at all. Loading screens seem to have 5-10% higher CPU usage, but load times are unchanged.

I wouldn't suggest to do that. I compressed my hdd to save space due to games and the result was that it was insanely fragmented. Loading times got so bad I had to format my hdd drive completely. Ever since then I am never compressing games again. Don't do it, it isn't worth it.

this is how bad it got
http://i.imgur.com/G18nTFC.png
http://i.imgur.com/951ubJB.png

Compact should not cause fragmentation like NTFS drive/folder compression set via Explorer as it does not operate in realtime.
Compression is a one-time thing rather than happening on-the-fly. If anything, using Compact is likely to reduce fragmentation.

Never considered it difficult since its just a right click operation.

That is not the same compression that Compact is using.

 

[Knurek]

Total Commander must be reporting "Size" rather than "Size on Disk". There's probably an option to change that somewhere.

Hollow Knight is similar. 9GB down to 1.65GB using LZX.
Game doesn't seem to be affected by this at all. Loading screens seem to have 5-10% higher CPU usage, but load times are unchanged.

He's not saying the reported size is wrong (although there's no option in TC to show Size on Disk) - he's saying that even after compression he doesn't get changes in free space reported.
(Which may be caused by CompactGUI not liking different codepages - I've been using the commandline version with no issues on a Polish Win10)

All Unity titles are showing great gains, Hollow Knight, Cosmic Star Heroine, Shadowrun games, etc. I guess the engine has no support for compressing datafiles...
Also, I don't think you should be using LZX for games - it has a more demanding decompressor than XPRESS method.

 

[FunkyDealer]

He's not saying the reported size is wrong (although there's no option in TC to show Size on Disk) - he's saying that even after compression he doesn't get changes in free space reported.
(Which may be caused by CompactGUI not liking different codepages - I've been using the commandline version with no issues on a Polish Win10)

All Unity titles are showing great gains, Hollow Knight, Cosmic Star Heroine, Shadowrun games, etc. I guess the engine has no support for compressing datafiles...
Also, I don't think you should be using LZX for games - it has a more demanding decompressor than XPRESS method.

The creator of this interface/compressor agrees with this on the github site. LZX isn't recommended for use on games and applications.

 

[Paragon]

Also, I don't think you should be using LZX for games - it has a more demanding decompressor than XPRESS method.

I generally have been trying XPRESS16K, but for a game like Hollow Knight which barely needs any CPU power and only seems to be loading on room transitions, rather than a 3D game which is constantly streaming in data, it doesn't seem to make a difference.
Anything where CPU performance matters, I've avoided compressing at all.

 

[Nabs]

Yes, but certainly not on this level.

Cuphead was a 1.9 gb download on Steam.

 

[FyreWulff]

Yeah, my bad. Was a bit tired when making the post.

The author has made another post on the Win10 subreddit, with a couple interesting stats: https://www.reddit.com/r/Windows10/comments/77scky/its_been_barely_a_week_since_i_posted_my/

Cuphead seems to handle compression really well, went from 11GB to 2.75GB. I wonder if this could be used to deliver games through Steam and the Win10 Store. If they could be downloaded heavily compressed as this and then unpacked on the HDD after, it would be great for people with datacaps.

and then someone with 5GB left on their HDD after downloading a 2.75GB game wonders why the game doesn't work because they had the space.

It would also mean a long wait. That's what the PS4 is doing when it does that forever "preparing download" setup, it's decompressing the entire game and then checking the bits so it can know where the patch data goes.

 

[snail_maze]

and then someone with 5GB left on their HDD after downloading a 2.75GB game wonders why the game doesn't work because they had the space.

It would also mean a long wait. That's what the PS4 is doing when it does that forever "preparing download" setup, it's decompressing the entire game and then checking the bits so it can know where the patch data goes.

But doesn't steam do this?

 

[wwm0nkey]

Wait.....has anyone tried DOOM?

 

[FyreWulff]

But doesn't steam do this?

Pretty sure Steam tells you the final size, not the download size, when you prompt to install a game.

 

[Nikana]

Wait.....has anyone tried DOOM?

This is a fantastic question.

 

[KainXVIII]

This is a fantastic question.

Why not to look in games list (op post have it)? Hint - its not great gain, but still.

 

[Plywood]

Wait.....has anyone tried DOOM?

https://github.com/ImminentFate/CompactGUI/wiki/Compression-Results:-Games

XPress16K Doom (2016) Before: 67 GB After: 51 GB Savings: 16 GB Compression: 76%
XPress16K DOOM 2016 Before: 62 GB After: 38 GB Savings: 24 GB Compression: 61%

 

[Slayven]

Thanks for the heads up, I needed a weekend project.

Is this for games? Or can do folders with videos, audio, and documents too?

 

[snail_maze]

Pretty sure Steam tells you the final size, not the download size, when you prompt to install a game.

Right, I think we are all talking about the same thing (compressed download) but the poster you responded to didn't know that steam already does this with downloads

 

[PumpkinSpice]

Thanks for the heads up, I needed a weekend project.

Is this for games? Or can do folders with videos, audio, and documents too?

Video and audio are already very compressed so you wouldn't see likely any savings. Most modern documents have at least gzip or zip compression as well.

Games have just gone in the direction of just not even trying to save disk space, which is why this is so effective in some cases despite being a performance oriented compression algorithm.

 

[data]

This thing is pretty unreliable with how much space it actually frees up on your HDD, like some other people mentioned. Sometimes it went down but not by the amount it actually compressed the folder by, sometimes it doesn't show more free space on my hdd at all.

 

[MissChief]

Keep in mind, playing heavy compressed games requires more processing power, so your pc uses more power. So you use more electricity and your hardware will have more wear .. basicly, after a few months, it’s cheaper buying a new HDD IMO

 

[Knurek]

Keep in mind, playing heavy compressed games requires more processing power, so your pc uses more power. So you use more electricity and your hardware will have more wear .. basicly, after a few months, it’s cheaper buying a new HDD IMO

(seriously, more wear? Also, do you think that new HDD doesn't need any power to run?)

 

[StrangeADT]

Uncompressing in real time (if that is what this does) would absolutely have an impact on performance. Longer load times and likely streaming issues.

The majority of time spent in loading is idle time for the CPU because hdds (yes, even SSDs, comparativey anyways) are so slow. There are reports that in certain cases this speeds up load time since the bottleneck is actually the loading of files into memory, so in some cases it is faster to load a smaller file and decompress it than to load an uncompressed file.

 

[Coonce]

Keep in mind, playing heavy compressed games requires more processing power, so your pc uses more power. So you use more electricity and your hardware will have more wear .. basicly, after a few months, it's cheaper buying a new HDD IMO

this is what the developer of the GUI says on the wiki of the github page regarding CPU usage

It's compression, so extra CPU power is required to decompress the program files as they're needed, however the XPRESS algorithm is highly efficient and designed for minimal CPU overhead. Any reasonable CPU will have minimal to no performance impact, and those using slower HDDs may even see a performance improvement as the smaller files can be transferred to the CPU much faster for processing. See the Microsoft doc here

it's a program made by microsoft with very well implemented compression algorithms.

in reality, your CPU will be minimally impacted, your SSD/HDD won't "die quicker" or anything like that.