Support NeoGAF

[Gekkonidae]

So since these boards are constantly embroiled in the argument of which console was better then which other console, I decided to do a little statistical analysis.

Using data from GameRankings and MetaCritic, I averaged the scores for the top fifty games on each console. This was under the reasoning that most people who own a console will not buy or play more than fifty games, and that a smart consumer should buy the best games available. Furthermore, I weighted the top games more heavily than the games further down the list, because presumably the best games are more important to the console's life than the fourty-ninth and fiftieth games. Furthermore, this gave a degree of statistical filtering that I can discuss later on if anyone cares.

Now, I'm not pretending that this is some kind of scientific survey, rather this is an interesting conversation starter and I believe it provides a good barometer of the quality of games on each system. We should also note that the oldest systems are probably at a bit of a disadvantage due to less reviews being published back then, and that new systems that are still getting games will certainly get better scores over time (I'll revisit these scores after the holiday season to see how the new releases have affected them.)

With no further ado, the scores:

Well, what do ya think guys?

 

[Speevy]

I think this young generation doesn't have enough games to make your already meaningless analysis valid.

 

[Hellraizah]

If Playstation 2 is Macho Man, Neo Geo is Mike Tyson.

 

[Aeris130]

N-Gage > Saturn?

 

[dallow_bg]

No SNES means you fail.

 

[Speevy]

N-Gage > Saturn?

The Wii is just slightly better than the N-Gage confirmed.

 

[platypotamus]

(I'll revisit these scores after the holiday season to see how the new releases have affected them.)

That's the biggest problem... small sample pool for the newer consoles makes your data heavily biased towards older systems... of course, only as far back as the reviews go. It would be less the case if you took the top x% of games I suppose, rather than an absolute number.

Also, there were 50 games for the N-Gage?

 

[Kuramu]

Intuition tells me that any list putting GC up that high is wrong

 

[ToxicAdam]

When our ancestors sift through the remains of our once proud civilization, they will see this data and realize we squandered all of our time on meaningless shit.

 

[Cheeto]

The Wii is just slightly better than the N-Gage confirmed.

Ps3 is just slightly better then the Wii confirmed.

 

[Speevy]

Remove all the systems that haven't gotten their last game.

 

[Woo-Fu]

I think your analysis is heavily weighted by size of library.

Size of library is important, but only up to a point. How many people buy 50 or more games for their console of choice? The only two platforms that meet that criteria for me are ps2 and pc, curiously enough.

I'd be interested in seeing how your numbers work using the top-25, the top-10, and an overall average.

 

[Gekkonidae]

No SNES means you fail.

Not enough data for systems that old. There just weren't enough reviews back then.

 

[Durante]

Well, except for the fact that consoles pre-PS1 are not represented and the fact that PC should be higher because of all the games released before online tracking the list seems to work quite well. Of course, using this methodology the position of current-gen consoles and handhelds will only ever move upward as new games (above each systems current composite score) are released.

 

[Gekkonidae]

I think your analysis is heavily weighted by size of library.

Size of library is important, but only up to a point. How many people buy 50 or more games for their console of choice? The only two platforms that meet that criteria for me are ps2 and pc, curiously enough.

I'd be interested in seeing how your numbers work using the top-25, the top-10, and an overall average.

Yes and no. There is a graded weighting system so that the top game is the most significant, and after that it trails off until the fiftieth game has very little weight at all.

 

[C4Lukins]

I would pretty much agree with the findings of the top 8. Of course there is no way the PS2 library is in anyway comparable with the entire PC library, but maybe for the time period which this encompasses it is accurate. The Next Gen systems are going to be lower because of the short period they have been around so that is dismissable. It seems about right to me.

 

[ElectricBlue187]

I approve of this list

 

[Stumpokapow]

Not enough data for systems that old. There just weren't enough reviews back then.

False. There were plenty of reviews, it's just hard to get a hold of the magazines necessary to copy the data over. It takes a very long time to transcribe this stuff and there just isn't the demand right now.

Around two years ago I wrote a site that downloads every review on GameFAQs and using bayesian weighting to compare across games and systems. You might say "GEEZ GAMEFAQS SUX" but at such a large sample size the effect of shitty reviews is lowered.

I might finish that site up and download all the data again.

Yes and no. There is a graded weighting system so that the top game is the most significant, and after that it trails off until the fiftieth game has very little weight at all.

Are you also using bayesian weighting to reflect the fact that games have differing numbers of reviews? You need to make sure to weight games with fewer reviews than the global mean towards the global mean score to compensate for the distorting effect few reviews will have?

 

[Tobor]

Not enough data for systems that old. There just weren't enough reviews back then.

No offense, but what's the point? You titled the thread "Greatest Console Ever". Not "Greatest Console Ever in the post internet era".

 

[soco]

i think this will end up like every other thread that attempts to analyze anthing.

people will start pointing out what they think are obvious mistakes and the thread will degrade into childish bickering until it's finally locked.

 

[Gekkonidae]

Remove all the systems that haven't gotten their last game.

Or just think of it as a snapshot of their libraries at this moment rather than an evaluation of how "good" the system is. Systems that are still getting better are still relevant to the discussion.

 

[ksamedi]

Its not fair that the PC is in there, it has many more games compared to individual consoles.

 

[ElectricBlue187]

No offense, but what's the point? You titled the thread "Greatest Console Ever". Not "Greatest Console Ever in the post internet era".

He titled it "Greatest Console Ever?(question mark) A statistical analysis'"

there are no statistics for that era therefore it is not included, why is this hard for you?

 

[Gekkonidae]

No offense, but what's the point? You titled the thread "Greatest Console Ever". Not "Greatest Console Ever in the post internet era".

You're right, I'm sorry the title is misleading. "Greatest console ever" refers not to my analysis, but rather to the constant bickering that my analysis is trying to address. Hence the use of a question mark.

Edit:

He titled it "Greatest Console Ever?(question mark) A statistical analysis'"

there are no statistics for that era therefore it is not included, why is this hard for you?

What he said.

 

[Kyosuke]

The Wii is just slightly better than the N-Gage confirmed.

But only because of GC ports. Without them, the Wii wouldn't stand a chance against the killer lineup of the N-Gage.

 

[Gattsu25]

How can you have a thread labeled the Greatest Console Ever and not include the Genesis in your graph?

 

[Gekkonidae]

Its not fair that the PC is in there, it has many more games compared to individual consoles.

True, but games that were made too long ago to have enough reviews are not represented. I think it sort of balances. And, again, this isn't really a scientific survey, so it's nice to have it for comparison.

 

[Durante]

Its not fair that the PC is in there, it has many more games compared to individual consoles.

What I find not entirely fair is that PC doesn't have all its games counted. Otherwise it would top this "Greatest Console Ever" list, which would be just delicious

 

[ElectricBlue187]

Its not fair that the PC is in there, it has many more games compared to individual consoles.

I would guess the only reason it's that high up is because most publications do not bother to review "Deer Hunter 12: The Revenge" and it's many clones

 

[Durante]

I would guess the only reason it's that high up is because most publications do not bother to review "Deer Hunter 12: The Revenge" and it's many clones

You clearly don't understand the methodology used. Once a system has 50 games, shovelware is irrelevant. (Personally I think that's a good way to do it, as it represents what an informed buyer can get out of the system)

 

[BirdBomb]

nice job, nice list.

im surprised the n64 is that low, the review lists for it are chock full of overrated nintendo titles imho.

 

[Tobor]

He titled it "Greatest Console Ever?(question mark) A statistical analysis'"

there are no statistics for that era therefore it is not included, why is this hard for you?

Why is it hard for you? This is like calculating the home run records in baseball by leaving out anything before 1960. It just doesn't give you an accurate analysis.

 

[ElectricBlue187]

You clearly don't understand the methodology used. Once a system has 50 games, shovelware is irrelevant. (Personally I think that's a good way to do it, as it represents what an informed buyer can get out of the system)

so it's the top 50 games?

Why is it hard for you? This is like calculating the home run records in baseball by leaving out anything before 1960. It just doesn't give you an accurate analysis.

Actually baseball is a fine example. Comparing Babe Ruth to Barry Bonds makes about as much sense as comparing the PS2 to the NES.

 

[Version 3.0]

Gamerankings is great for last gen (PS2, Xbox, Gamecube, etc). Outside of that, it's pretty useless.

And even then, data vs. reality is pretty iffy. Your assumption of people playing the best-rated games is hilarious. Check sales and rentals against scores and you'll quickly find otherwise. And your assumption of <50 games is wrong for both the PS2 and Gamecube, for me personally. Also, if you're weighting better-rated games heavier, you should rate exclusives heavier.

But really, this exercise is pretty useless. If you want to know the "hive mind" opinion of which console is best, look at sales.

 

[Gekkonidae]

so it's the top 50 games?

Yes. Although it's weighted so that the top games count more than the ones farther down the list.

 

[ksamedi]

The Xbox must be the most overrated console ever, i sort of agree with the PS2 though, the greatest console since the SNES.

 

[Durante]

so it's the top 50 games?

Just read the OP. It's described clearly and exactly -- well, except for the weighting factor used.

 

[Campster]

So we're neglecting historical importance (Where the hell are Atari systems?) or systems that have a few absolutely amazing titles surrounded by crap, and choosing only the top 50 games ever (giving an insane bias to platforms with the biggest libraries). And then claiming we've achieved an objective answer to the age-old "What is the best console ever?" question, which is just such a sad and fanboyishly competitive concept that I just hang my head.

Smrt.

 

[The Abominable Snowman]

At the very least, switch the position of the GC and DC.,

 

[Gekkonidae]

Just read the OP. It's described clearly and exactly -- well, except for the weighting factor used.

The weighting is simple: where n represents the rank of the game from 1 to 50 and s is its average review score, the total score is the sum of (51 - n)*s for each n divided by 1250.

So the top game has a weight of 50 and the 50th game has a weight of 1. I ran this algorithm independently for gamerankings and meta crtitc and averaged those two scores for the total.

 

[DeaconKnowledge]

SNES and Genesis not there makes this entire list shit.

 

[Gekkonidae]

And then claiming we've achieved an objective answer to the age-old "What is the best console ever?" question

I never claimed that. I'm only trying to add something to a discussion that in the absence of real analysis can only ever be "a sad and fanboyishly competitive concept."

 

[Campster]

SNES and Genesis not there makes this entire list shit.

Or Atari systems, or the Apple II, or Commodore 64, Gameboy, Game Gear, Sega Saturn...

Look, the whole concept is stupid, biased, ignorant of all history of the industry, trying to put an objective spin on an inherently subjective question, and all about fanboy wankery.

 

[ElectricBlue187]

The weighting is simple: where n represents the rank of the game from 1 to 50 and s is its average review score, the total score is the sum of (51 - n)*s for each n divided by 1250.

So the top game has a weight of 50 and the 50th game has a weight of 1. I ran this algorithm independently for gamerankings and meta crtitc and averaged those two scores for the total.

So if a system has ten perfect 10 games and fourty 2.0 games it might beat a system has has consistent 6.5 games?

I guess that's what I didn't understand. That would clearly favor systems with great first party games and systems that have been out for awhile but if you counted Deer Hunters and such, it would introduce outliers, gotcha.

 

[urk]

It's a decent idea, but it fails on a number of levels. Primarily, current generation systems do not have a full catalog of titles to represent their top 50 titles yet. The 360, for example, only has about 280 retail titles outside of XBLA. That means the top 50 titles will invariable include some stinkers. Same with the PS3 and the Wii. The low number of releases makes this comparison void.

 

[Tobor]

so it's the top 50 games?



Actually baseball is a fine example. Comparing Babe Ruth to Barry Bonds makes about as much sense as comparing the PS2 to the NES.

This is a mine field you've stepped into that I don't intend to follow.

A question mark in and of itself does not do enough to validate the results, plain and simple.

"Greatest Modern Console Ever" would have been fine.

Or better yet, just like Barry Bonds, put an * next to the PS2. Problem solved.

 

[dallow_bg]

Not enough data for systems that old. There just weren't enough reviews back then.

Yeah, I realize that.

Just the thread title says Greatest Console Ever.


Should have an asterisk. Like Barry Bonds.

 

[cartoon_soldier]

How much time did you spend?

Did you play Bioshock Demo yet?

 

[joesmokey]

Something tells me we're going to see a "Your favorite console ever?" thread after this...

Anyway, pretty pointless OP. Great job of saying "Greatest Console Ever?" and having the earliest console in the list be the Playstation/Dreamcast/N64 era.

Spoiler

Btw, the PC isn't a console, nor are handhelds for that matter.

 

[Gekkonidae]

Or Atari systems, or the Apple II, or Commodore 64, Gameboy, Game Gear, Sega Saturn...

Look, the whole concept is stupid, biased, ignorant of all history of the industry, trying to put an objective spin on an inherently subjective question, and all about fanboy wankery.

This is no more "an objective spin on an inherently subjective question" then the concept of numerical review scores is. I simply took a larger sample of reviews and tried to produce a list that compares the relative strength of library of various systems. Its only wankery if you want to wank about it. If you want to have an actual discussion, then it's not.

Again, I'm not ignorant of the awesomeness of systems before the PS1. There just isn't enough data for older systems. I'm sorry there isn't, now get over it. We can still discuss them, I just can't produce numbers for them.