Support NeoGAF

[QuickSilverD]

This is something I've been wondering about for some time now. Since PS4 save file data takes up a minimum of 10.5MB, which to me sounds like too much and I understand most of it is probably just padding for some hardware reason, but still, if a game has multiple save slots, then each one takes up that much space, especially comical with a (mostly linear) adventure game like Life is Strange, that uses 3 save slots for the game and on top of that 3 for the options for those, totaling over 60MB save in total.

But even if we look at a save file that is not padded, like for example Bloodborne, which is a single save file (I only have one character) is over 30MB big. am I am here wondering, why? what exactly is being kept track of here? isn't that what save files are? simple documents with checks that say what you where doing? for Bloodborne it should be tracking your coordinates in the map, which items you have and how many of them, which story triggers are set and which boss killed, your stats and so on, so how come all that needs anything more than 100KB?

The angle I'm looking at this is, if I were to track everything that needed to be tracked in Bloodborne and I used a word document to keep track of everything, that document would probably not be more than a few tens of pages long and weight like 200KB tops, which makes me think that it isn't just markers and raw information being save, but something else.

Does anyone have any technical explanation for this phenomenon?

 

[Athreous]

Because nowdays games have more stuff to be saved, bigger inventory, location data, etc =p

Remember Diablo 1 on PS1? it used a whole memory card if you wanted to save your quests =D

 

[LogicStep]

I had some Dragon Age Inquisition save files that were taking around 400-500MB. It gets pretty crazy.

 

[ghostjoke]

Think of every single choice/variable/decision you made in a game, from your stats, the weapons you chose and their upgrades, the order you did certain objectives and how they effected each other, to the simple did you pick up a certain item or miss it. Now try to write that out into 1s and 0s.

 

[Skittles]

I think you're just under-estimating how much data a game actually stores nowadays.

 

[Vincent Grayson]

I think the answer is simple: because they can be.

 

[Garibaldi]

The irony of the whole thing is a bet a fair few save files are bigger than the executable that the game uses to run. You're right though. I have no idea why they need even 10MB to save. It's surely just GUIDs for resources with extra metadata (which itself cannot be very large) as well as trigger markers for checkpoints/positional information etc

Think of every single choice/variable/decision you made in a game, from your stats, the weapons you chose and their upgrades, the order you did certain objectives and how they effected each other, to the simple did you pick up a certain item or miss it. Now try to write that out into 1s and 0s.

It's not as big as you'd think. An integer value is at most 64bits or 8 bytes

 

[Sir_Crocodile]

The storage allows for it so devs don't have to be tricky about it anymore.

 

[SolidusDave]

Because nowdays games have more stuff to be saved, bigger inventory, location data, etc =p

Remember Diablo 1 on PS1? it used a whole memory card if you wanted to save your quests =D

No, that is not the reason.

Today's games don't need 100-1000 times more space to save progress. Especially not compared to games of the previous gen, there isn't that much of a change in complexity.


The 10 MB minimum thing is also something we don't really understand about the PS4.
You can compare the save files of various cross-gen games that are literally the same, yet the PS4 save file is 10-100x bigger. (and the data on the PS3 is already including some art work etc.).


In many cases the PS4 save files aren't even actually that big, they simply reserve that space.
Try to upload a save file to the PS+ cloud and you will notice that it goes faster than your actual internet upload speed would be capable of (because only the actual data is transferred).

 

[Athreous]

No, that is not the reason.

Today's games don't need 100-1000 times more space to save progress. Especially not compared to games of the previous gen, there isn't that much of a change in complexity.


The 10 MB minimum thing is also something we don't really understand about the PS4.
You can compare the save files of various cross-gen games that are literally the same, yet the PS4 save file is 10-100x bigger. (and the data on the PS3 is already including some art work etc.).


In many cases the PS4 save files aren't even actually that big, they simply reserve that space.
Try to upload a save file to the PS+ cloud and you will notice that it goes faster than your actual internet upload speed would be capable off (because only the actual data is transferred).

Sorry mate, but for a ton of games it is, specially rpgs.

 

[Laconic]

Stuff.

 

[QuickSilverD]

I think you're just under-estimating how much data a game actually stores nowadays.

Maybe, I figure that by variable you mean even something like. [item in world A at coordinates xxxyyyzzz picked: yes/no] which should take what? 50Bytes tops in a lousy format? that's still 20,000 variables per MB.

Are save files really just raw data? is what I am asking

 

[DOWN]

Shouldn't save data be tiny raw data? There just aren't that many variables added this gen to justify the jump.

I'm going with they are bigger just because they can be.

 

[Jimrpg]

Maybe Sony set the size at 10MB across the board. One thing u don't want is file sizes getting bigger as u play the game... There might be a situation where u run out of space and unable to save the game which would suck.

 

[SolidusDave]

Sorry mate, but for a ton of games it is, specially rpgs.

But it can't be to the extend we are seeing. Maybe open world games like Fallout 3 which save every item's position and what not but even that is like 15 MB on PS3 with all addons installed.
There isn't a jump in complexity compared to the previous gen that would lead to all games having bigger save files.
FF7 managed with 67 KB...


Anyway, I mainly wanted to point out that on PS4 (the 10 MB minimum mentioned in the OP) there is another factor as we can compare the save files of cross-gen games and still find vastly different sizes.
If they reserve the space for each game to ensure there's enough room in case the save file grows, fine, it's still a stupid way as the majority of games will only ever need a fraction of that and it annoyingly uses up the space of the PS+ cloud.

 

[Lelcar]

In Skyrim you can store any item in any of the thousands of nooks and crannies of that game and those items will stay there forever. I think even if you drop something on the ground at a random spot, it will stay there until you pick it up again. That combined with the other variables that folks mentioned above makes saving every little detail a much bigger task.

 

[duck_sauce]

Because people want choices and open world experiences.

 

[Give War A Chance]

This seems like it'll be a short thread. Pretty simple answers. First post nails it. There's just way more to be saved now.

 

[gruenel]

Sorry mate, but for a ton of games it is, specially rpgs.

Name a PS4 RPG that is much more complex than Skyrim or Fallout 3? Should be easy if there are "tons."

This seems like it'll be a short thread. Pretty simple answers. First post nails it. There's just way more to be saved now.

Except there really isn't.

A Diablo 3 savegame on PS3 is about 150KB.
A Diablo 3 savegame on PS4 is >10MB.

Explain that if it isn't some kind of useless padding.

WWE 2K15 has an 800MB+ save file on PS4. There's a big difference between "more stuff to save" and plain stupid.

This, and I bet the same save game would be less than 10MB on PS3.

 

[Adda]

WWE 2K15 has an 800MB+ save file on PS4. There's a big difference between "more stuff to save" and plain stupid.

 

[Fuz]

Because nowdays games have more stuff to be saved, bigger inventory, location data, etc =p

They don't.
Proof: Daggerfall or Ultima VII.
They had a shitload of things to track in saved games.

It's just lazy optimization on the devs side.

 

[jgwhiteus]

Think of every single choice/variable/decision you made in a game, from your stats, the weapons you chose and their upgrades, the order you did certain objectives and how they effected each other, to the simple did you pick up a certain item or miss it. Now try to write that out into 1s and 0s.

I feel like RPGs of every generation have had to save the same data, and in fact some older RPGs (on both PC and console) would be considered "bigger" than many modern games by those standards. But I don't remember the save files being nearly as big. And it's not like in previous generations you could only have 100 items in your storage, but now you can hold 1,000,000 items - game design in certain genres has remained relatively stable; it's only the presentation elements (graphics / audio, etc.) that have changed, which shouldn't affect save data.

I don't understand it either, OP. Maybe there are just default platform-specific files that have to be included with every save for encryption / decryption, etc.

 

[Laconic]

Name a PS4 RPG that is much more complex than Skyrim or Fallout 3? Should be easy if there are "tons."

Skyrim and Fallout 3 are prime examples of save file bloat.

Hell, as you went on, the save files blew up like 2000s Steven Segal.

 

[infinityBCRT]

The save files don't need to be that big. Reason's why they are big?

- tighter deadlines for today's games means they can't go back and optimize stuff like save files (or ship a game that isn't broken!)
- it's easier just to dump everything in memory to a save file, because it will probably just work, and there's no thought necessary into trying to pair down what is saved vs what isn't, nor would you have to do any extra work to create the state of the game if pretty much everything is saved

 

[Gaming Truth]

I think the answer is simple: because they can be.

The storage allows for it so devs don't have to be tricky about it anymore.

Shouldn't save data be tiny raw data? There just aren't that many variables added this gen to justify the jump.

I'm going with they are bigger just because they can be.

This. There's really nothing much more complex in today's games that require more storage. There's just no need to take the extra hours/days to come up with an algorithm to efficiently compress save data for space and time concerns.

Back in the good ol' days, when we either had limited storage (cartridges) or patience (PS1 saves, good god), it was definitely worth doing some planning to reduce the footprint. Something akin to:

4-bit representation of weapon ID
8-bit representation of the weapon's XP
1-bit flag for whether mod #1 is on it or not
etc.
etc.

A Call of Duty save should never be bigger than 1 MB if done right (excluding tag pics). "Choice" games like Mass Effect should be the same.

 

[gruenel]

Skyrim and Fallout 3 are prime examples of save file bloat.

Hell, as you went on, the save files blew up like 2000s Steven Segal.

That is exactly my point, Skyrim and Fallout 3 have bloated save files and they STILL need less space than even the simplest PS4 games. Hell a Pix the Cat savegame on PS4 is bigger than most of my Skyrim saves on PS3.

 

[Koren]

I think you're just under-estimating how much data a game actually stores nowadays.

I think you're over-estimating it...

Since memory isn't an issue anymore, most developpers don't try to optimize the save files for space anymore. Many are nearly a memory dump, or a serialization of every variables in memory.

Out of curiosity, I tried to estimate the memory needed for a Disgaea save. The file size should be 20-30 times smaller at the very least (probably several times smaller since I didn't even took into account possible compression of data). Fun fact about this game: items aren't stored as IDs, the actual *name* of the item is stored. Should you change the language of the game in the middle of the game, you'll have items in different languages.

 

[Rosur]

Lack of optimisation from devs I think is the main reason as they just use whatever the engine puts out for the save file.

 

[WhirlwindMonk]

Think of every single choice/variable/decision you made in a game, from your stats, the weapons you chose and their upgrades, the order you did certain objectives and how they effected each other, to the simple did you pick up a certain item or miss it. Now try to write that out into 1s and 0s.

A walkthrough for 100%ing a game basically does exactly that, plus includes stats of every item and monster along with explanations of everything you need to do in human-readable format, yet the largest walkthrough on gamefaqs for FFX is under 3 MB. Compress it and it'll be even smaller.

There are roughly 500,000 integers in 1 MB, and the majority of the stored bits of info don't need a variable anywhere near that large. There are few games outside of Minecraft that need to store that many pieces of data.

10 MB is insane for pure, numerical data. Just like software itself, the file sizes are probably bloated because the space is there so the devs don't have to bother being efficient anymore.

 

[QuickSilverD]

WWE 2K15 has an 800MB+ save file on PS4. There's a big difference between "more stuff to save" and plain stupid.

really, isn't that a fighting game? does it has some sort of loot systems where every item needs to be store with coordinates or something

Skyrim and Fallout 3 are prime examples of save file bloat.

Hell, as you went on, the save files blew up like 2000s Steven Segal.

I'm not familiar with these games save file structures, how big can those save files get?

 

[jett]

Because without restrictions some developers stop caring about optimization.

 

[sturgboski]

The next question is: if save files this gen are so large, why is PS4 cloud storage so woefully small?

 

[LonelyGreyWolf]

Now try to write that out into 1s and 0s.

Lol, that's not quite how this works.

 

[astrogamer]

really, isn't that a fighting game? does it has some sort of loot systems where every item needs to be store with coordinates or something

It could be storing the created wrestler and various amount of career progression though I can't imagine it being that big without a few of the former

 

[EoinOL]

Maybe Sony set the size at 10MB across the board. One thing u don't want is file sizes getting bigger as u play the game... There might be a situation where u run out of space and unable to save the game which would suck.

Sony definitely set a minimum file size of something a bit over 10MB for PS4 saves. Even games that have separate save files for storing the settings (which should be a few KB at most) create ~11MB save files.

Running out of space and not being able to save would be a good reason for Sony to have set a minimum size, but the size they've chosen is too big - they should have either gone for a small size, or at least made ~11MB the default and then allowed developers to elect to have a smaller size if they knew for sure that there they wouldn't ever need so much space.

 

[Koren]

Think of every single choice/variable/decision you made in a game, from your stats, the weapons you chose and their upgrades, the order you did certain objectives and how they effected each other, to the simple did you pick up a certain item or miss it. Now try to write that out into 1s and 0s.

It's pretty easy, actually.

Think here's 2 billion possible actions/events in a game (that's a huge overestimate)

Think you're playing for 100h and do an action every second (that's also a huge overestimate)

Even if you store this as a huge log without any compression (which is really unefficient), the complete log would be 1.5MB.

And 99% of the log is useless in the save...


As far as item picked/not picked is concerned, even if there's 15.000 items in the game, the availability of those require less than 2kB.

 

[FranXico]

Does anyone have any technical explanation for this phenomenon?

It's related to the PS4 file system. Files that have copy protection (note that you can only copy your saves to/from USB while logged in to PSN) have such a large size as a side effect of the encryption.

To me, it is far worse that certain game cache files are also listed as saves and end up taking all of my online storage space. The worst offenders actually are first party titles (SingStar, LBP3, etc.). I find myself having to turn auto-upload off and micro-manage files.

 

[Fragamemnon]

Being efficient w/ save file sizes has little impact on customer satisfaction of the product, and doing things like writing your own compression serializers/derserializers for game objects or making specific new structures just to store save file data and have intermediaries to export and import the data into usable engine representations is just not worth it. It's high risk code for very little reward.

It's not so much laziness as it is not stepping into a minefield. The more straightforward and "close to the engine" the save file data is, the less chance you introduce a bug that gets by QA that corrupts save states. The tradeoff is that the data is going to be pretty large on disk.

The next question is: if save files this gen are so large, why is PS4 cloud storage so woefully small?

The small sizes don't make a lot of sense outside of the overall general pain of upgrading the capacity for the end users. I don't think it's Sony being cheap but just the glacial pace that t infrastructure upgrades take in some places.

 

[Percy]

Dunno, but it's a massive pain in the ass on PS4 seeing as you only get 1gb for cloud storage on saves. That's a worthless size nowadays.

Sony definitely set a minimum file size of something a bit over 10MB for PS4 saves. Even games that have separate save files for storing the settings (which should be a few KB at most) create ~11MB save files.

Don't think that's it as Stardust Ultra's saves are well under 10mb for one. It's confusing as shit why pretty much everything else seems to be working to this minimum though.

 

[ThoseDeafMutes]

The next question is: if save files this gen are so large, why is PS4 cloud storage so woefully small?

I assume that they won't bother upping it until people make a fuss. Maybe twitter faux outrage will get it done.

Perhaps they can rent some Azure servers if they don't have the infrastructure for it. Actually they could just register a new OneDrive account for every PS+ subscriber and get 15 GB of cloud storage for free.

 

[MiketheBSG]

Name a PS4 RPG that is much more complex than Skyrim or Fallout 3? Should be easy if there are "tons."



Except there really isn't.

A Diablo 3 savegame on PS3 is about 150KB.
A Diablo 3 savegame on PS4 is >10MB.

Explain that if it isn't some kind of useless padding.



This, and I bet the same save game would be less than 10MB on PS3.

Can someone give me a response to this? That pretty much seems like padding to me.

 

[hobblygobbly]

I've written save file/load management in the past. There's no universal way of doing it, everyone does it differently. The most common approach is simply serialisation.

Basically the more you need to keep track of, the more you need to serialise.

However this depends, you can either make a conscious effort of knowing what you should serialise and what you shouldn't. A lot of people don't have the time or don't care about it so they serialise literally everything and then load that data back in. You can write code to serialise entire objects out as-is, a common approach.

But lets say you are smart about your serialisation and care about the size and what you serialise, this requires more time and effort. Lets say you are playing a RPG with random loot spawns and you enter a dungeon. Maybe that dungeon spawned 24 items of different things, this all followed an algorithm/RNG system to do that to begin with and that had a seed value. Say the player decides to save in the dungeon before clearing it. Should I serialise out each item + its accompanying data (such as its stats, its location, whatever else) or should I just serialise out the integer of the seed value? One would serialise out more data and not gain anything while if you just serialise out the integer its literally bytes. Randomisation in computing and procedural algorithms everything is governed by a seed and we use a seed to replicate the output if we so wish. If you use the see 42 in Minecraft it'll always generate the same world for example.

If it's a proper algorithm/generator and you give it the same seed it'll always reproduce the same thing EXACTLY the same so there's no reason to keep track/serialise out its stats/position in the dungeon if the algorithm will reproduce the exact same output again juts with the seed - the same stats, the same location, everything. So why would I need to serialise all of the individually spawned items in the dungeon if I could just serialise out the seed value used to generate it initially + keep track of the item IDs that were looted already? When the player loads that save file all you do is call the algorithm that generated the items, feed it the seed and tell it to exclude the item IDs that you kept track of (which would just be integers) as looted and it'll respawn them. The downside on that is then maybe the loading time will be a little longer since it'll call the algorithm again. Or would it? Maybe loading all that serialised data would maybe take 1-2 longer. Who knows, but if you did it smartly you reduced the size of what should be tracked by 99%.

Basically it depends on the type of game, how it's architecturally built and whether programmers want to spend time consciously optimising save files. Chances are they won't and will take the easy approach and call it done since there are higher priority things to focus on. Again I just want to reiterate that saving/loading in games is one of those things in the industry that has zero widely accepted implementation, everyone does it differently.

 

[Costia]

Maybe they are now storing the entire state of the game and not just the diff from the "clean slate".
If you have to conserve space you can only save the things the player has changed since he started playing. Things like moved item locations, choices etc. Then you would have to run an algorithm that looks at the current state and the "clean" state and saves the differences. Now if you have a game patch that changes what the initial state is, your save file might get broken because it relies on the pre-patch state.
Without space limitation you could just dump the entire game state and restore it on load. This could contain things the the player has never encountered or changed, for example the location and state of all items - even those the player have never seen. So such a save would be a lot larger. Now since everything is stored there would be no need for developing and running a "diff" algorithm which could mean less bugs and faster loading and save times. This will also avoid having a growing save file which will ensure that saving will never fail due to lack of space (on the hard drive or the cloud).

 

[Caayn]

Except there really isn't.

A Diablo 3 savegame on PS3 is about 150KB.
A Diablo 3 savegame on PS4 is >10MB.

To make it even stranger Diablo 3 savedata is 10KB on the XB1.

 

[Winterfang]

Think of every single choice/variable/decision you made in a game, from your stats, the weapons you chose and their upgrades, the order you did certain objectives and how they effected each other, to the simple did you pick up a certain item or miss it. Now try to write that out into 1s and 0s.

That happened to game last Gen. A saved state shouldn't take that much space,is not new content is just a key to unlock content on the disc.

 

[QuickSilverD]

hobblygobbly, can you explain what simple serialization is? in this context, I'm not too well bested on the technicality of it all

Maybe they are now storing the entire state of the game and not just the diff from the "clean slate".
If you have to conserve space you can only save the things the player has changed since he started playing. Things like moved item locations, choices etc. Then you would have to run an algorithm that looks at the current state and the "clean" state and saves the differences. Now if you have a game patch that changes what the initial state is, your save file might get broken because it relies on the pre-patch state.
Without space limitation you could just dump the entire game state and restore it on load. This could contain things the the player has never encountered or changed, for example the location and state of all items - even those the player have never seen. So such a save would be a lot larger. Now since everything is stored there would be no need for developing and running a "diff" algorithm which could mean less bugs and faster loading and save times. This will also avoid having a growing save file which will ensure that saving will never fail due to lack of space (on the hard drive or the cloud).

Wouldn't this increase loading times?

 

[Oni Jazar]

This seems like it'll be a short thread. Pretty simple answers. First post nails it. There's just way more to be saved now.

Incorrect.

To reiterate (thanks to Caayn & gruenel):

A Diablo 3 savegame on PS3 is about 150KB.

A Diablo 3 savegame on XB1 is 10K.

A Diablo 3 savegame on PS4 is >10MB.

It's a) how the developer chooses to save the data and b) how the system chooses to handles saves in their API.

The GB cloud limit on PS4 wouldn't be bad if Sony had stricter save requirements.

 

[Koren]

It's related to the PS4 file system. Files that have copy protection (note that you can only copy your saves to/from USB while logged in to PSN) have such a large size as a side effect of the encryption.

There's definitvely a side-effet, that won't triple the size, let alone x100 it (and it's definitively x1000 or x1000 sometimes)

Being efficient w/ save file sizes has little impact on customer satisfaction of the product

Overall, I agree, but with limitations in cloud saves, that's no more exactly true.

It's not so much laziness as it is not stepping into a minefield. The more straightforward and "close to the engine" the save file data is, the less chance you introduce a bug that gets by QA that corrupts save states. The tradeoff is that the data is going to be pretty large on disk.

Totally agree, but that's only because you design the engine first (without thinking memory usage), and saving is an afterthough. Should you try to be clever with memory in the engine, the save files will also be greatly reduced.

I think, personally, that you shouldn't design an object in a game engine without thinking at the same time how you will serialize it.

Maybe they are now storing the entire state of the game and not just the diff from the "clean slate".

Even if you're storing the entire state, if you're doing it correctly, it won't be more than a couple MB for the most complex games (except games where the user actually create content).

This could contain things the the player has never encountered or changed, for example the location and state of all items - even those the player have never seen. So such a save would be a lot larger.

In several cases, that can be actually smaller. Should you keep a list of picked items, you have to store a list of items IDs. Should you keep a list of the status of all items, you can store it as a list of bits, so 16 or 32 times less bits.

As soon as the user has picked more than 3-6% of the items, the entire state takes less place than the diff.


That being said, many developpers use simple booleans tables for item status (instead of bit collections, which uses far less memory, but is a bit less straightforware to use in many languages), so it's not always as simple as this.

 

[The Albatross]

There's more room, less need for compression, more ability to save everything that a game wants to save. Back even on the PS2, you were limited to an 8 MB memory card, so games had to compress a lot and weren't able to truly save what they wanted to.

WWE 2K15 has an 800MB+ save file on PS4. There's a big difference between "more stuff to save" and plain stupid.

Jesus... This is ridiculous especially considering there's, what, a 25 CAW limit? Compare this to like PSX Smackdown which let you create like 75 guys with a full suite of creation tools and fit everything onto a 1 MB memory card.

 

[mackattk]

If the save file is under 20MB I don't really care too much. If it gets over 50MB+ it starts to be a problem. Especially with the PSN cloud only being 1GB.

WWE2K15 being 800MB+ is fucking insane.