Support NeoGAF

[I AM JOHN!]

I think you better look up the definition of 'average'...

Average - adjective - of the usual or ordinary standard or level of quality.

As in, it's not actively bad, but it doesn't excel in any way, shape, or form. So tell me how I'm wrong, friend.

 

[McBradders]

I guess the question I would throw back to you is--why would they ignore such a powerful entity? It's one thing to be "different" but it's another to be oblivious. Perhaps they are making things more confusing (on a comparison basis) in an attempt to avoid accountability or scrutiny? Not saying that's the case, but reasonable minds could infer.

Also I think the dismissal of metacritic is a bit overblown on here. Just cue up their top or bottom games and I think most who have played them will agree that metacritic can provide useful information. Especially to those who don't have the time to read multiple reviews or really narrow down which sites they deem trustworthy for certain genres, platforms, etc. It's a great normalizer of all the drama and bias that non-gaffers just don't have the time to parse out.

Reminds me of when Garnet flipped out on 1UY when Dyack was just trying to say look at some of the top games on metacritic, some don't sell well. As if citing metacritic for this simple purpose is some heinous offense.

My problem with Metacritic is the arbitrary occlusion of results that can't be quantified in numerical form, or that it goes ahead and converts non-numerical data in to numerical data for the sake of its system using a fully undisclosed method (if I'm incorrect about the method being "secret" I apologise in advance as it makes my entire point moot :3). That, to me anyway, makes it untrustworthy. At least with something like Rotten Tomatoes, it's user input that makes or breaks the rating, and as such is far more valuable and accurate resource.

It's the same thing with VGC's magic secret formula of hogwash that just makes it completely untrustworthy.

And it doubly angers me that some publishers take this data to heart and reward/punish accordingly.

 

[Stoney Mason]

I think you better look up the definition of 'average'.

While he isn't exactly kind about it, he has a point. Because the average score of an average is 5.0 doesn't mean exactly half the scores will fall above that mark and half will fall below that mark. Or that even when you average all their scores it will become a 5.

Generally people are sticking to the notion of 5 being the concept of an average game in a theoretical sense. If they give a bunch of games above 5 for one month that doesn't mean the notion of an average game increase because good games pull up the average. It just means there were an unusually high number of above average games for that month. Average is a fixed concept.

Of course this tends to fall apart and degrade the scale when they consistently throughout the year, over-rate games or are not consistent. This isn't aimed at any one site btw. Just a general observation.

 

[Crushed]

75% is a D? Since when?

crushed has yet to move beyond high school.

either that, or crushed is implying the average gaffer hasn't moved beyond high school.

either way works.

Oh come off it. 75% falls into "D" for many school's grading systems.

 

[LiveFromKyoto]

We switched to letter grades because we felt that it'd be more universally understood than our previous numeric scale. You see, in the past, we insisted on using the entire 0-10 range, with 5 being in the middle, rather than an "8-10 is good, everything below that is bad" scale most other outlets seem to use. But just because we insisted on it didn't mean all our readers bought into it. So while a 7 meant "not bad, still good" to us, it looked like it was on the verge of failing to some in our audience. We tried it for a few years and decided it wasn't working, so we switched to a letter-grading scale that we knew would be crystal clear to everyone.

Here's the reality - everybody always rags on this, but a game getting less than a 7 is a failing grade. There are more quality games out there than any sane person can ever play, so unless something hits a sweet spot for your tastes, there's no reason to go for anything below the above average-great range.

 

[Stoney Mason]

There are more quality games out there than any sane person can ever play, so unless something hits a sweet spot for your tastes, there's no reason to go for anything below the above average-great range.

I completely disagree although of course it is a subjective call. There is so much out there I would never buy and is very average imo. I think scores are on a whole are too high for videogames as a medium.

 

[FordStang]

At first I thought, "Oh God, that's awful..."

Then I realized that this is actually a great move. Most gamers just seem to take their letter grades from school ("7.5!? You mean that game got a D!?").

Seeing a B instead of an 7.5, or an F instead of a 2 or 3, makes much more sense and conveys the quality of the game more.

You do know a 7.5 out of 10 is a C, right?

 

[Phobophile]

Hopefully this letter grading system is so vague that people will start paying attention to the review itself rather than the score.

Ah fuck, who am I kidding?

 

[commish]

You do know a 7.5 out of 10 is a C, right?

I think someone missed the entire point of the conversion

You are the exact kind of person who EGM thinks can't understand their old review system.

 

[pizzapizza64]

I think this is a great idea and something that other companies should consider.

 

[Stoney Mason]

Hopefully this letter grading system is so vague that people will start paying attention to the review itself rather than the score.

Ah fuck, who am I kidding?

If gamers don't read manuals, they sure as hell aren't going to start reading reviews.

 

[OnPoint]

If gamers don't read manuals, they sure as hell aren't going to start reading reviews.

Gamers are too tough for manuals.

 

[duk]

i couldn't care less about this

 

[D.Lo]

As in, it's not actively bad, but it doesn't excel in any way, shape, or form. So tell me how I'm wrong, friend.

Average is a statistics term for the arithmetic mean. Other uses of the word are metaphors applying the mathematical term to non-mathematical qualities.

You want to claim non-mathematical use of numbers?

And as I said in my second point you carefully avoided, they claimed they were better then other sites because they used the whole scale while others only used the top end. Well statistically their use of scores is nearly identical to their main competition. Gamespot actually has a lower average score then them.

 

[LiveFromKyoto]

I completely disagree although of course it is a subjective call. There is so much out there I would never buy and is very average imo. I think scores are on a whole are too high for videogames as a medium.

I don't disagree with you, I like the tough way Edge grades because as it stands the 80% range for most outlets currently means "graphics are decent and the disc doesn't explode," 80 is par for the course for anything with a decent budget that doesn't make any major missteps. I think we'll eventually see review outlets move to the 4 star system, since they can just slap a 2 on the meh stuff and nobody gets mad. "Hey, it's only one dot away from being really good, and two dots away from perfection!"

But I do think that there are at least 10-15 really good games released a year, which is more than I can really get through - a lot of them have timesink multiplayer components, and the sports games & lengthy rpgs take a while for anybody who isn't in school.

 

[Flavius]

The question here really is: Does gaming really need all these intellectuals like Shane and Garnet pontificating about these "issues" from their ivory towers? Apparently the entire elitist Ziff Davis family thinks that the answer is "yes," but I for one do not appreciate being lectured to. If I want to go to a high-level philosophy class, I'll go back to college.

Personally, I love those guys, but...no, just no. :lol

 

[I AM JOHN!]

While he isn't exactly kind about it, he has a point. Because the average score of an average is 5.0 doesn't mean exactly half the scores will fall above that mark and half will fall below that mark. Or that even when you average all their scores it will become a 5.

You know, I should probably be apologizing for the Rage Jizz I've shot all over this thread, but fuck it, I'm not, because it really fucking pisses me off how much Metacritic and other aggregate sites have perverted the word "average." When has average, in terms of art criticism, ever meant "all our scores added together" before the rise of the aggregators?

Not that it really matters, since Aggregators aren't doing their job as it is. How is Game Rankings giving you a good idea of the quality of a game when they place a 7 from EGM, a 7 from IGN, and a C- from Gaming Age, roughly translated to 70%, next to each other, when every single one of those scores has a different meaning to it? Or Metacritic, when they do the same thing, but add a subjective slant onto it? The only aggregator that does it right is Rotten Tomatoes' movie division: they give you a pull-quote and a basic pictorial summary that says either the reviewer liked it or didn't like it. That's how aggregates are supposed to be, not this faux-mathematics bullshit.

Average is a statistics term for the arithmetic mean. Other uses of the word are metaphors applying the mathematical term to non-mathematical qualities.

You want to claim non-mathematical use of numbers?

Fuck yes I do, because the way they're being presented, these numbers have no meaning. See above, fool.

And as I said in my second point you carefully avoided, they claimed they were better then other sites because they used the whole scale while others only used the top end. Well statistically their use of scores is nearly identical to their main competition. Gamespot actually has a lower average score then them.

I didn't carefully avoid it, it just doesn't matter. Average score doesn't mean "THIS IS WHAT THE MATHEMATICAL AVERAGE IS;" it means "THIS IS WHAT THE SUBJECTIVE AVERAGE" is. How many fucking times do I have to say this?!

I'm not even going to address your argument that "THEY SAY THEY'RE BETTER THAN THESE SITES" because it's obvious that you just have a vendetta against the Ziff Game Group.

 

[dog$]

I think the dismissal of metacritic is a bit overblown on here. Just cue up their top or bottom games and I think most who have played them will agree that metacritic can provide useful information.

All the top list of Gamerankings proves to me is that there are a handful of titles which few people can form any abject criticism towards or otherwise feel compelled to give certain games higher ratings than they personally might feel they deserve.

Let's que up GR's Top 10 at the moment:

Title Plat Company Reviews Avg.
Vote Avg
Score
1. The Legend of Zelda: Ocarina of Time N64 Nintendo 31 9.0 97.613%
2. Super Mario Galaxy WII Nintendo 65 8.9 97.302%
3. Metroid Prime GC Nintendo 91 8.8 96.264%
4. Soul Calibur DC Namco 27 8.8 96.259%
5. The Orange Box PC EA Games 30 9.6 96.183%
6. The Orange Box X360 EA Games 51 9.1 96.114%
7. Super Mario 64 N64 Nintendo 21 9.2 95.905%
8. Resident Evil 4 GC Capcom 102 9.2 95.827%
9. Tekken 3 PS Namco 23 8.1 95.804%
10. Resident Evil 4 PS2 Capcom 55 8.7 95.673%

All this list proves to me is that there are a handful of games which there are few detractors out there who are willing to do anything but brainlessly agree that these 10 particular games are somehow better than others, their own personal taste be damned.

I can personally break the list down against my tastes. I don't care about platformers, so SMG and SM64 are out. I don't care about FPSes either, so Metroid Prime and Orange Box is out. I can think of 10 fighting games I'd rather play than Soul Calibur or Tekken 3. I've always hated how the Resident Evil series has controlled and 4 is no different. If I were to review these 10 games, I'd maybe give Orange Box a 7 or 8 - I've never played it. The rest I could comfortably give 7s and lower.

When the new DDRs or Dynasty Warriors roll in, reviewers are all too happy to hand out their 5s with the caveat tacked on of how the games are "good if you're a fan of the series". Where's a review that gives Super Mario Galaxy a 6 for that exact same reason?

I'd be willing to bet at least 1 of any 2 people who were to buy that set of 10 Games would wind up disappointed with what they have. Gamerankings doesn't know my taste, or yours, or anyone's, so to say that those 10 games are something which a majority of people should have is simply absoultely wrong. All GR shows is that a majority can be wrong.

Sites that compile all of this information in one easy to use score renders their individual contributions to the entire medium of "game journalism" pointless to say the least. It means that these individuals with clear biases can no longer influence the market the way they have in the past.

You are giving game journalists far too much credit here.

Biases ... wash out when viewed as a whole.

Which is unfortunate, because biases are more realistic and applicable on an individual basis to how people approach their purchases of videogames and which ones they like. I don't care how many millions of people will buy SSBB and write reviews of unending praise for the game - I know I wouldn't want to buy it, so I won't buy it, and no amount of praise from those millions will convince me otherwise.

If anything, there should be more bias in journalism. It would let people know that they don't have to convince themselves to like something which they don't because "everyone else does" and that it's acceptable to actually have an independent opinion on something. It would also result in reviews actually being more convincing and engaging.

 

[KGKK]

My mind is blown!

 

[Lykathea]

This is a step in the right direction, that direction being eliminating scores entirely and letting the text of a review speak for itself.

 

[anachronous_one]

*reads the short quote*

the real question is will people accept C as average just because they do IRL, or will they translate the letter grades to anything less than a B sucks?

People accept C's? I felt like dogshit the few times that I got anything worse than a "B".

And I'll also mention my new personal mantra on reviews --- as someone else has already noted here --- "Thumbs up/Thumbs down, with a buy/rent/avoid recommendation ftw."

 

[Walshicus]

Numbers>Letters

Latins > Arabs

 

[LukeSmith]

I give this decision an 8.5.

But what letter grade would you give it? Is that B or an A- or a B+ in this brave new world?

 

[I AM JOHN!]

Romans > Arabs

Well duh, Christians > Muslims

And by that, I mean Americans > Terrorists

I'M SORRY, I THOUGHT THIS WAS AMERICA!

EDIT: lol i c wut u did thar. Doesn't change the joke.

 

[Crushed]

You do know a 7.5 out of 10 is a C, right?

You do know that there is no standardized grading system that all schools follow, right?

 

[commish]

But what letter grade would you give it? Is that B or an A- or a B+ in this brave new world?

Fuck man, don't convert that shit! That's why EGM isnt' releasing their conversion algorithm!!!

 

[Walshicus]

Well duh, Christians > Muslims

And by that, I mean Americans > Terrorists

I'M SORRY, I THOUGHT THIS WAS AMERICA!

EDIT: lol i c wut u did thar. Doesn't change the joke.

Heh, yeah I thought people might think I was talking about Roman Numerals vs Arab Numbers rather than the Latin Alphabet vs...

 

[JJConrad]

What I love about EGM's rants about gamers not getting their review scale is that is that even they subconsiously recognize a 70% as average (its evident in their review articles and average scores), but they have this need to stomp their feet and call out millions of gamers for being too stupid to understand that a 50% isn't a bad score. It's cute, in a pathetic way.

 

[Stoney Mason]

Not that it really matters, since Aggregators aren't doing their job as it is. How is Game Rankings giving you a good idea of the quality of a game when they place a 7 from EGM, a 7 from IGN, and a C- from Gaming Age, roughly translated to 70%, next to each other, when every single one of those scores has a different meaning to it? Or Metacritic, when they do the same thing, but add a subjective slant onto it? The only aggregator that does it right is Rotten Tomatoes' movie division: they give you a pull-quote and a basic pictorial summary that says either the reviewer liked it or didn't like it. That's how aggregates are supposed to be, not this faux-mathematics bullshit.

While I'll agree that Gamerankings and Metacritic could be improved and the examples you pointed out are such and there could be a lot more, I would argue that most big games that get reviewed by a majority of the big sites across the web receive the same bias. Therefore comparing scores across games isn't that useless because they are all receiving the same treatment. When I go look at a movie review at Gamerankings or Rotten Tomatoes, personally I do the same thing. I look at the score. I compare it to what else is currently out and then I drill down and find the reviewers that I actually trust and see their score. These sites are consumer tools imo. If publishers misuse them, then that's on them.

 

[Classic_Gs]

Numbers>Letters

I agree. I wonder if 1up/egm made this change due to pressure from publishers for low scores (i.e. Assassin's Creed). I guess a D looks better than a 4.5 and helps them avoid gamerankings and such.

 

[I AM JOHN!]

While I'll agree that Gamerankings and Metacritic could be improved and the examples you pointed out are such and there could be a lot more, I would argue that most big games that get reviewed by a majority of the big sites across the web receive the same bias. Therefore comparing scores across games isn't that useless because they are all receiving the same treatment. When I go look at a movie review at Gamerankings or Rotten Tomatoes, personally I do the same thing. I look at the score. I compare it to what else is currently out and then I drill down and find the reviewers that I actually trust and see their score. These sites are consumer tools imo. If publishers misuse them, then that's on them.

It's a huge difference, though. Rotten Tomatoes looks at every review, tells you who said what, and then gives you a number based on the number of reviewers who liked it vs. the number of reviewers who didn't.

GameRankings and its ilk, however, compile every review number, regardless of scale or meaning, and average it into one lump sum, giving you a number that doesn't have any meaning.

 

[Elbrain]

Well at first I thought it was a bad idea but then after some thought it think it really good. Screw with the system and the machine!

 

[Scotch]

Nice for Americans, I guess. For most of the rest of the world, a letter grade system isn't easier to understand at all.

And this isn't gonna change anything. You will still hear complaints that a "C+ is too low".

 

[Dali]

Why put a letter? Why not just right a review and let the reader make a decision as to how the written review translates as far as a score? A letter grade is really no different than a numerical score and it will still spurn the same type of debates about whether game "X" was an A- while game "Y" was a B+ and all the conspiracy theories as to why included.

A lot of people suggest this, but I think a summary of one's thoughts (using a number or letter or whatever) in addition to a descriptive review is ideal. Words can be ambiguous. A reviewer can say one thing about a game and then backtrack or say he/she was misunderstood. A score is a concrete metric that a reviewer must stand by and leaves the writer in the position where the only wiggle room he has in respect to explaining a review is within the context of the A or F he gave a game.

Nice for Americans, I guess. For most of the rest of the world, a letter grade system isn't easier to understand at all.

And this isn't gonna change anything. You will still hear complaints that a "C+ is too low".

You are at least the second person to make this comment. EGM is an American magazine. When they say "universally understood" they mean within the target readership.

 

[Moz La Punk]

Dunno if someone figured it out in this thread (I didnt read through it), but the one interviewed has some major issues.

We're not publicizing the conversion scale because we want our readers to go with our new scoring system and not be constantly translating the new letters back to our old scores.

Ok, great. So what's this all about?

But most people can figure it out. Our old "average" in the 5 range roughly translates to the C letter grades (with plusses and minuses), for example.

 

[kbear]

Great move by EGM and 1UP.

 

[camineet]

EGM Volume 1, Number 2 - July 1989 - page 6

First of all, we've changed our reviewing format. Do you get sick of reading reviews in other magazines that sound like someone rewrote the manual? How about critics that pretend to have something to say, only to rave about each product in the final paragraph of their review? Well, with our new multi-reviewer format, you get the compete picture from four of the hottest players in the nation as well as at-a-glance number ratings that tell it like it is! We're promising to give you plenty of solid, up-to-date reviews (not games that are three years old) with info that will help you buy the games that are right for you.

It's been swell EGM.

 

[The Faceless Master]

seeing how this thread turned out already, the change to letter grades has already gotten a big fat F

 

[Xisiqomelir]

I think you better look up the definition of 'average'...

Looking at the shittiness of a typical game would be more beneficial, imo.

 

[SaggyMonkey]

Could someone also figure out what writers even mean by "average" now? It's a pretty useless term given the relative quality in genres.

Like what's an average music game right now... Rock Band, Guitar Hero, Pata Pon, AudioSurf, Sing Star, DDR? The "average" music game seems like it's pretty good then right? So is average here more like an 80 or 85?

What's the "average" shooter that people buy and play right now? COD4? Half Life 2? Bioshock? Halo 3? Team Fortress 2? Portal? Turok? ETQW? UT3? Resistance? Gears of War? Lost Planet? Warhawk? Again, it seems like "average" shooter player is a really frikkin good game (85+?). Then obviously you'd see games like Hour Of Victory, Blacksite, Turning Point, etc. occupying the "below average" area.

Using the term "average" to mean "5 out of 10" or "C" is nonsense, as it would seem like "average" is a constantly changing term for every style of game.

 

[I AM JOHN!]

Could someone also figure out what writers even mean by "average" now? It's a pretty useless term given the relative quality in genres.

Like what's an average music game right now... Rock Band, Guitar Hero, Pata Pon, AudioSurf, Sing Star, DDR? The "average" music game seems like it's pretty good then right? So is average here more like an 80 or 85?

What's the "average" shooter that people buy and play right now? COD4? Half Life 2? Bioshock? Halo 3? Team Fortress 2? Portal? Turok? ETQW? UT3? Resistance? Gears of War? Lost Planet? Warhawk? Again, it seems like "average" shooter player is a really frikkin good game (85+?). Then obviously you'd see games like Hour Of Victory, Blacksite, Turning Point, etc. occupying the "below average" area.

Using the term "average" to mean "5 out of 10" or "C" is nonsense, as it would seem like "average" is a constantly changing term for every style of game.

What does the lack of an "average" game in a genre matter?

Stop with the fucking fake math. It doesn't actually mean anything!

 

[Naeblish]

Someone has been playing too much dmc4 :lol .

 

[Scotch]

You are at least the second person to make this comment. EGM is an American magazine. When they say "universally understood" they mean within the target readership.

You do know EGM is being sold outside the US, right? Not to mention 1UP which everyone in the world can read.

I know that the American readership probably dwarves everyone else, but saying it's universally understood is just not true.

 

[FirstInHell]

The question here really is: Does gaming really need all these intellectuals like Shane and Garnet pontificating about these "issues" from their ivory towers? Apparently the entire elitist Ziff Davis family thinks that the answer is "yes," but I for one do not appreciate being lectured to. If I want to go to a high-level philosophy class, I'll go back to college.

You and I seem to really be on the same wavelength. I do not mind listening to discussions about gaming as a topic, but it the way that people like Shane, Garnett, and N'Gai wax poetic about certain ideals just comes across as condescending and overly pretentious. It seems that they get on their soapbox with the "high horse of the month. This month, the high horse is NARRATIVE. This month it is going to be REVIEW SCORE INTEGRITY. I listen to a podcast because it is gamers like myself talking about games. I do not need to be educated by the likes of N'Gai as he drones on with his theories about what what equals good game design from a philosophical standpoint. He presents his opinions like they are so groundbreaking and epic when he is really not saying anything that is either relevant or no overly obvious.

 

[B-Rad Lascelle]

It's a huge difference, though. Rotten Tomatoes looks at every review, tells you who said what, and then gives you a number based on the number of reviewers who liked it vs. the number of reviewers who didn't.

GameRankings and its ilk, however, compile every review number, regardless of scale or meaning, and average it into one lump sum, giving you a number that doesn't have any meaning.

It wasn't too long ago that Rotten Tomatoes had a Tomatometer-score to track the review consensus of weekly game rentals. Problem is that almost everything scored a "Fresh" rating so it had to be dropped.

Your system doesn't work with games due to the tendency of many reviewers to softball scores to cater to the whims of game publishers. If they had the freedom to be as critical about games as movie reviewers are of movies, then it would be an aggregate scoring approach worthy of more consideration and merit.

But not today.

 

[Stoney Mason]

You do know EGM is being sold outside the US, right? Not to mention 1UP which everyone in the world can read.

I know that the American readership probably dwarves everyone else, but saying it's universally understood is just not true.

There is clearly an American Bias with a A thru F letter grading system, but it is what it is.

 

[Scotch]

You and I seem to really be on the same wavelength. I do not mind listening to discussions about gaming as a topic, but it the way that people like Shane, Garnett, and N'Gai wax poetic about certain ideals just comes across as condescending and overly pretentious. It seems that they get on their soapbox with the "high horse of the month. This month, the high horse is NARRATIVE. This month it is going to be REVIEW SCORE INTEGRITY. I listen to a podcast because it is gamers like myself talking about games. I do not need to be educated by the likes of N'Gai as he drones on with his theories about what what equals good game design from a philosophical standpoint. He presents his opinions like they are so groundbreaking and epic when he is really not saying anything that is either relevant or no overly obvious.

I'm glad I'm not the only one who thinks this way. N'gai is so overrated.

 

[I AM JOHN!]

FirstInHell and Astrolad, I'm so glad you're in here shitting this thread up because you hate that everyone in the Ziff Game Group treats you like you have intelligence.

It wasn't too long ago that Rotten Tomatoes had a Tomatometer-score to track the review consensus of weekly game rentals. Problem is that almost everything scored a "Fresh" rating so it had to be dropped.

Your system doesn't work with games due to the tendency of many reviewers to softball scores to cater to the whims of game publishers. If they had the freedom to be as critical about games as movie reviewers are of movies, then it would be an aggregate scoring approach worthy of more consideration and merit.

But not today.

I agree with you, but that's another discussion entirely.

 

[Propagandhim]

EGM changes to a letter grading system and GAF intones with elaborate philosophical manifestos on the nature of mathematics.


Par for the course!

 

[FirstInHell]

Isn't it kind of hypocritical how EGM will not disclose its magic formula for converting game scores to numbers, yet they lambaste Metacritic for not disclosing their secret formula?