• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Developers losing source code / asset

W

warheat

Member
I've heard stories that some of the parts from a remake/remaster for example Kingdom Hearts, Silent Hill, and maybe few others has to be rebuilt/rewritten from scratch because apparently the developers no longer have the source code/asset or it is lost.

Source :
https://en.wikipedia.org/wiki/Silent_Hill_HD_Collection
http://www.eurogamer.net/articles/2...on-still-uses-ps2-cut-scenes-in-separate-ways
http://www.ign.com/articles/2013/06/27/original-kingdom-hearts-assets-lost
http://www.the-nextlevel.com/tnl/threads/52361-Developers-Losing-Source-Code

Notice that this mostly happen to Japanese developers. For some background, I worked for a few companies in the past 5 years, even the small one at least have weekly backup. One of the company I worked for still have their code from the 90's.

So I'm wondering how the hell this is even possible?
 
N

Noobcraft

Member
I heard that the recent HD version of Heroes of Might and Magic 3 didn't include the expansions because the source code was lost for them. Not sure if that's true, but since 3DO went under it seems plausible.
 
N

Napalm_Frank

Member
IIRC the KH thing was really overblown, they reverse engineered it from disc and supposedly that is perfectly fine way to do it (Bluepoint did it for all their titles AFAIK).

Funny thing about SH collection is that they didn't do this, they used old code which probably didn't help.

The whole ''rebuilt from scrach'' is total bullshit on any of these tho.
 
S

Shahadan

Member
Big companies, changing offices, moving stuff somewhere and forgeting what is is, malfunctions, etc

Things can get lost.
 
nkarafo

nkarafo

Member
I heard the original arcade Bubble Bobble source code is lost.

Another reason why emulators that can run these games are important.
 
K

KZXcellent

Member
Napalm_Frank said:
IIRC the KH thing was really overblown, they reverse engineered it from disc and supposedly that is perfectly fine way to do it (Bluepoint did it for all their titles AFAIK).

Funny thing about SH collection is that they didn't do this, they used old code which probably didn't help.

The whole ''rebuilt from scrach'' is total bullshit on any of these tho.
Click to expand...

Yeah for some reason they decided to not grab a retail copy of Silent Hill 2 and just used an older build of the game. That build had glitches/issues they had never seen before and made that collection even worse.
 
U

UnNamed

Banned
Sega had lost lots of original code from genesis and saturn era. That's why they never remade games like Panzer Dragon or Daytona, which was only emulated on PS360.
 
K

Katori

Member
Napalm_Frank said:
IIRC the KH thing was really overblown, they reverse engineered it from disc and supposedly that is perfectly fine way to do it (Bluepoint did it for all their titles AFAIK).

Funny thing about SH collection is that they didn't do this, they used old code which probably didn't help.

The whole ''rebuilt from scrach'' is total bullshit on any of these tho.
Click to expand...
As a software engineer, I can tell you that reverse engineering it "from the disk" sounds like a living nightmare. Especially if the game uses some kind of encryption--it might not even be a possibility. It's cool that they pulled it off to great effect with KH, though.
 
K

KJRS_1993

Member
I think the development of the Sims 2 had a problem with a fire or flooding or something?
I don't remember 100% so could just be making it up!

Edit: Just Googled, think it was a fire
 
K

Krosia

Member
Half-Life 1.


Their central asset storage died close to release and they had to piece the game from the devs pcs together.
 
C

cubicle47b

Member
It's insane that it ever happens, and I'm skeptical when it does (when the project is crowdfunded), because secure, professionally administered online source control repositories exist and they're absurdly cheap compared to labor.

It's somewhat understandable pre-2008 or so, but why aren't there at least off site backups of your repo?
 
N

Napalm_Frank

Member
Katori said:
As a software engineer, I can tell you that reverse engineering it "from the disk" sounds like a living nightmare. Especially if the game uses some kind of encryption--it might not even be a possibility. It's cool that they pulled it off to great effect with KH, though.
Click to expand...

AFAIK the team behind SH never gave info why they didn't do it. I think there was some interview or something from Bluepoint where they talk about why they reverse engineered their titles.
 
Lister

Lister

Banned
There's no excuse in this day and age. Code is usually up on distribute source code management server in the cloud and local resources are always backed up on premises and off-site.

It's almost like Japanese dev's have never heard of an IT department.

Of course things would have bene different in the past.
 
T

The Albatross

Member
cubicle47b said:
It's insane that it ever happens, and I'm skeptical when it does (when the project is crowdfunded), because secure, professionally administered online source control repositories exist and they're absurdly cheap compared to labor.
Click to expand...

Yeah I can see this back in the 90s when centrally managed source control would have still been kept on site, but since the mid 2000s any organization maintaining their own source control that doesn't have absurd levels of redundancy is Busch league.
 
K

KingBroly

Banned
Capcom and Mega Man Mania, the Game Boy Mega Man games. At least, that's what I remember them saying back in the day. They had no problem finding code for the Virtual Console, though.
 
Strangelove77

Strangelove77

Member
I'm guessing back then they didn't think they'd be porting games generations ahead. You made the game for all current systems and called it a day.
How many "late" ports existed back then? Off the top of my head(though I'm sure I'm wrong) I didn't play an "enhanced" port til the GBA with all those Super Mario and Final Fantasy games.
I don't think preservation for games was an idea they had back then.
 
Theonik

Theonik

Member
Archival is relatively expensive and for these companies going back to these products was not on the cards until decades later.
 
L

laxu

Member
I'd guess a lot of these happened before version control systems became popular and back then it must have been entirely due to not making sure you have a backup procedure in place. Incompetence or cost cutting basically.
 
gingerbeardman

gingerbeardman

Member
Mousnis said:
I heard the original arcade Bubble Bobble source code is lost.

Another reason why emulators that can run these games are important.
Click to expand...
That's the most famous one I know of.

Also the reason why no "home" version of Bubble Bobble is anywhere near the quality of the arcade version.

tsundoku said:
No Man's Sky?
Click to expand...
Has there ever been confirmation that they started the game again from scratch after the flood loss?
 
-hadouken

-hadouken

Member
Much of the code for Garou Mark of the Wolves II was purportedly lost/stolen in the naughties. What I wouldn't do to have that game finished and released!
 
R

rayman1900

Member
I can never forgive sega to have lost the source code from Panzer Dragoon Saga. It will be remembered only by few of one of unique jrpg of its kind to ever existed...
 
F

funkystudent

Member
Wasn't the source code for the original prince of persa lost for decades until it was found in Jordan Mechners garage?


Then he uploaded it to github
 
R

R.D.

Neo Member
I can kinda see this all as being normal, especially for japanese developers. I don't think they used a lot of Version Control back in the days, which is the standard to use these days when it comes to code and even assets.
Theses system allow to store code and assets on servers and share them between devs making sure that eveyone can have the newest code and can update the server with via commits of new code. These system are pretty powerful and I can't see myself ever working without them. You can even rollback to the earliest stages of development and so on.

Since I never heard of companies from Japan being using those systems, I can kinda see how they "lost" the code, as I'm sure they also had restrictions on what a programmer can take home. From today's perspective it's hard to not be in awe about this and wonder how this could ever happen.

I also wanna say that "reverse engineering" is not a simple task. It's hell, pure hell. Not to mention having to dive into code that you didn't write. And the worst of all: Not having a unified code style and language. I've seen code where comments are in all languages, but never English. Makes it really hard to work on. I understand it for languages with different letters/symbols like Japanese or Russian, but not e.g. German/French and so. Can't even understand what the comment says and having to figure out by reading the code is such a waste of time. That is assuming someone even left a comment to begin with as a lot of programmers tend to be lazy when it comes to that (me included to be honest)
 
C

Chev

Member
warheat said:
Notice that this mostly happen to Japanese developers. For some background, I worked for a few companies in the past 5 years, even the small one at least have weekly backup. One of the company I worked for still have their code from the 90's.

So I'm wondering how the hell this is even possible?
Click to expand...
A regular backup doesn't even begin to address the problem. Backups are there to address short term failures, not archival. You can't freely leaf through the backups to find something, restoring one is an expensive process, so you have to already know exactly what, when and where you are searching for.

Which is why you need some proper archival, through repositories, and then you hit on the fact that to be effective as archives, those repositories have to be known and handled at the company level in some way. You may have wonderful source control and it won't be even a bit useful if all the people who knew about it are gone or in places where none of their successors will think of asking them.

This also works for physical assets, like how a Capcom employee discovered an extremely useful style and anatomy guide for fighting games had been written by the Capcom golden age artists, and then forgotten during the post-SF3 period until he found a copy in a company warehouse.

I've personally had the experience of changing teams, and even though there was a repository, and I had instructed people before leaving, and there was an intranet-wide search engine that'd allow them to find the repository anyway, it only took four months before no one knew where the repo was, no one wanted to search for it (or even knew the search engine would work), and the new programmer thought it easier to start his new, own repository based on the latest copy of the code he had because he wasn't very good with git and couldn't figure it out and let's make a SVN guys. The company prided itself on giving freedom to their programmers so no one thought of checking whether he was doing the right thing.

I've also had the case where an employee dutifully tried to format the dedicated machine where a repository was because he thought there only were his personal, ahem, photo archives on the server and he wanted to clean up before leaving (and for the same reasons, he had taken the server out of the global backup schedule and no one had paid attention). Thankfully we got to that one in time.

And when a project's coming to an end, especially if it was a rough end, and it often is with video games, no one wants to dedicate time and resources to archival. They only want to be gone on vacation, and once they come back management want them to be 100% on the next project.

The takeaway is that you have to dedicate people to archival, and companies often don't because they'd rather have them do work they think is more useful. There's no real incentive to it if short term is more important.
 
5olid_5nake

5olid_5nake

Member
Are there any good articles about reverse engineering in games? Or in general?

I would absolutely love to read in detail how Bluepoint and other developers work their magic.
 
B

Borman

Member
Here's the thing, lots of developers do archive. They've had some sort of source control for a while. That's gets backed up to tapes and that sort of thing.

Great, right? Well then they move studio, a new guy comes in, whatever. Then no one brings the backups along so they get thrown out. Sometimes that gets saved by someone like me, most of the time not.

Then there is the issue with maintaining. Backups are good, but you need to monitor them and keep them accessible. CDrs and the like are rotting, especially poor ones from the early 90s. Heck, I've had PS2 era data with bitrot. Then those tapes, they tend to explode when stuff goes wrong, taking out the drive and the tape. Data lost.

Then there is accessing the data even if the media is safe. If no one has kept a log of what was used to back things up, and how to restore things once the data is accessible, that adds another layer. Plus you hope that they saved the dev environment with all those little customizations they made.

Yeah, its tough. I've restored a few dev environment, but it takes a lot of time and hope that everything is there.
 
Sapiens

Sapiens

Member
I hold out hope that developers will eventually find this stuff on old machines/floppies/discs they might have taken home with them.

It's a real shame when we live in a word where a shitty Wordpress plugin has a GitHub repository and 500 branches and Panzer Dragoon has nothing.
 
C

Costia

Member
Could be due to copyright/licensing issues as well.
I have been part of a company acquisition. Lots of code got dumped because the licenses weren't clear or non-existent.
Sometimes, as far as the legal team is concerned, its easier/cheaper/safer to rewrite things than to deal with possible lawsuits later on.
 
A

ArchAngel

Member
rayman1900 said:
I can never forgive sega to have lost the source code from Panzer Dragoon Saga. It will be remembered only by few of one of unique jrpg of its kind to ever existed...
Click to expand...

I have the hope of someone at SEGA finding a disc with the code sometime just used as a table stabilizer.
 
V

v1oz

Member
oBa said:
Sega losing source code is an understatement, they in most cases never keep it. At a previous company I worked for, Sega had contracted us to do a pack of their retro titles. When asked for the source code for the titles (approximately 75 genesis and sms titles) Sega responded with "we don't have source code for any of those projects". and even better they suggested we "go out on the internet and download the roms from various rom sites".
Click to expand...

I just had to quote this after all the talk of Panzer Dragoon!
 
keraj37

keraj37

Member
Yep, even Blizzard lost its source for Star Craft 1.
But there is happy ending: it was found this year and returned to its parents:)

EDIT:
I keep my source of my projects always in online repositories, so losing for good my code equals end of the world or simply coincidence of malfunction every server hardware on which it physically resides (usually clouds have many copies across many servers for CDN like distribution)
 
L

levious

That throwing stick stunt of yours has boomeranged on us.
UnNamed said:
Sega had lost lots of original code from genesis and saturn era. That's why they never remade games like Panzer Dragon or Daytona, which was only emulated on PS360.
Click to expand...

Daytona is on psn/xbla is emulated?
 
H

HotHamBoy

Member
UnNamed said:
Sega had lost lots of original code from genesis and saturn era. That's why they never remade games like Panzer Dragon or Daytona, which was only emulated on PS360.
Click to expand...


The source code for House of the Dead 1 has been lost for decades.

Sega fuckin sucks at holding on to their shit.
 
ethomaz

ethomaz

Banned
Backup are not forever too... you need to check them frequently... so think about backups in some obscure room in the company for 10 years or more.

And Japan has natural disasters too...

Today backup is cheap but 10-20 years ago it was really a hell of expensive... a ZIP drive was expensive and could allow about 500MB backup... there is magnetic tape backup too.
 
B

BetterThanNothing

Member
Yohane said:
Didn't Half Life 2 source code got stolen at some point during development?
Click to expand...

Yea, it was a big deal. Completely affected the way they continued onward with that game from what I recall, because the hacker was leaking assets and source code to the internet.

They ended up catching the guy because Gabe Newell actually contacted the guy and told him he wanted to offer him a job. Guy flies out to US thinking this is true, gets arrested as soon as the plane lands.

Don't fuck with Gabe.
 
X

xam3l

Member
Pinnacle Station, a DLC for Mass Effect 1 only exists for PC and Xbox. The PS version does not exist due to data corruption, and the team couldn't building it.
 
keraj37

keraj37

Member
Martin5000 said:
Does Nintendo backed up the source code from their NES and SNES games?
N64 seems so because of Oot an MM 3D
Click to expand...

Even if not, taking in account that NES compiled binary exe build was something around 2-16kb there was not much lost so it would be fairly easy to rewrite it all using modern languages instead of assembly.
 
Liliana Eileen

Liliana Eileen

Member
BetterThanNothing said:
Yea, it was a big deal. Completely affected the way they continued onward with that game from what I recall, because the hacker was leaking assets and source code to the internet.

They ended up catching the guy because Gabe Newell actually contacted the guy and told him he wanted to offer him a job. Guy flies out to US thinking this is true, gets arrested as soon as the plane lands.

Don't fuck with Gabe.
Click to expand...

Holy shit! Whatever happened to that guy in the end?
 
M

madjoki

Member
BetterThanNothing said:
Yea, it was a big deal. Completely affected the way they continued onward with that game from what I recall, because the hacker was leaking assets and source code to the internet.

They ended up catching the guy because Gabe Newell actually contacted the guy and told him he wanted to offer him a job. Guy flies out to US thinking this is true, gets arrested as soon as the plane lands.

Don't fuck with Gabe.
Click to expand...

iirc, he didn't make it to planet as he was arrested by Germans first, but plan was to arrest in US.
 
T

Tain

Member
One of my favorite tales of source code mishandling is Konami including the compressed source code to beatmania 5th Mix as a dummy padding file on the beatmania Best Hits PS1 disc.

Not quite source code loss, but even more spectacular.
 
You must log in or register to reply here.

Similar threads

  • Locked
IGN: How Hidden Nazi Symbols Were the Tip of a Toxic Iceberg at Life Is Strange Developer Deck Nine
3
656
tkscz replied
IGN: A Year Since Its Release, Sony Seems to Have Abandoned PlayStation VR2
4 5 6 7 8
355
17K
CamHostage replied
Another Code: Recollection | Review Thread
Analysis Review Hype
28
2K
Thick Thighs Save Lives replied
[IGN] Ex-Rockstar Dev Sheds Light on Long Lost Game Agent's Development Troubles
25
2K
StueyDuck replied
IGN’s GOTY 2023 is The Legend of Zelda: Tears of the Kingdom
Opinion 2 3
124
6K
Hudo replied
Top Bottom