• Hey Guest. Check out your NeoGAF Wrapped 2025 results here!

Creating a new "metric" for reviews.

I got a crazy idea. I can't tell if it's good or perhaps the worst review idea ever. That's how crazy it is.

I propose a Top Gear rating system.

Starting TODAY, games are ranked respective to each other. If a site starts with Uncharted 2 for example, it would be rated 1st place, because there are no other games ranked yet.
Brutal Legend next. They give a review, then in the end compare it to Uncharted. "BL has some great moments, but overall not as great as Uncharted 2." Ranked #2.

Modern Warfare 2 is next up. "GOTF. I'll be playing this for years." Takes the #1 spot.
Game X follows. "Not as good as MW2. Better than Uncharted 2." Takes the #2 spot.

Now we have:
1 - Call of Duty: Modern Warfare 2
2 - Game X
3 - Uncharted 2
4 - Brutal Legend

Continues from then on. Which game will take down MW2? FIND OUT NEXT EPISODE!

Potential problems:
1. No hard metric. A fast lap in Top Gear gives you a lap time. No arguing there.
2. Same roundtable of reviewers have to conduct the review each time. If guy A reviews game X, then how can guy B rank game Y higher?
3. Fanboy rage guaranteed. Awesome for site hits, high potential for blatant exploitation though.

This may work better for a podcast rather than an actual review site, but yeah that's my idea.
 
None.

If you don't feel like reading an excellent review, you will read after the next game you failed to be able appreciate and know why.

"But my friends liked and I know I should like it too... but I don't? Help?"
 
The whole idea of ratings is to give a game a specific number on a scale, to represence an objective comparability. Of course this is dumb because you can't be really objective about something and also game A and game B aren't comparable. But thats just how the business roll at this moment. The whole system just hasn't evolved to how it is in the film industry by now. Best you can do as a videogame "journalist": Use as few gradations as possible so i.e. game A and game B can both recieve a 3 of 4, although you prefer one of them, but at the same time you admit that way that one can think the other way around.
 
ChefRamsay said:
I got a crazy idea. I can't tell if it's good or perhaps the worst review idea ever. That's how crazy it is.

I propose a Top Gear rating system.

Starting TODAY, games are ranked respective to each other. If a site starts with Uncharted 2 for example, it would be rated 1st place, because there are no other games ranked yet.
Brutal Legend next. They give a review, then in the end compare it to Uncharted. "BL has some great moments, but overall not as great as Uncharted 2." Ranked #2.

Modern Warfare 2 is next up. "GOTF. I'll be playing this for years." Takes the #1 spot.
Game X follows. "Not as good as MW2. Better than Uncharted 2." Takes the #2 spot.

Now we have:
1 - Call of Duty: Modern Warfare 2
2 - Game X
3 - Uncharted 2
4 - Brutal Legend

Continues from then on. Which game will take down MW2? FIND OUT NEXT EPISODE!

Potential problems:
1. No hard metric. A fast lap in Top Gear gives you a lap time. No arguing there.
2. Same roundtable of reviewers have to conduct the review each time. If guy A reviews game X, then how can guy B rank game Y higher?
3. Fanboy rage guaranteed. Awesome for site hits, high potential for blatant exploitation though.

This may work better for a podcast rather than an actual review site, but yeah that's my idea.
Click to expand...


Personally, I would prefer a “Cool Wall” approach, if we’re going to base ourselves off Top Gear.

No group consensus, complete anarchy and placement based more on image than execution.
 
I don't mind the current system. I think it's fine. I think Buy, Rent, Don't Buy, is far too generic and broad, and doesn't help you identify the really good games from good games. Most people only have time to play the really good games, so I for one am glad for the current ranking system.

I do agree that reviews should be much shorter. Two paragraphs is all I need.
 
I struggle to think of gaming as an industry unique in that there's a problem with the scoring system, rather than the scorers themselves. If there's anything wrong with reviews, the problem is more likely to do with the reviews and not the arbitrary numbers attached to them at the end.
 
ChefRamsay said:
I got a crazy idea. I can't tell if it's good or perhaps the worst review idea ever. That's how crazy it is.

I propose a Top Gear rating system.

Starting TODAY, games are ranked respective to each other. If a site starts with Uncharted 2 for example, it would be rated 1st place, because there are no other games ranked yet.
Brutal Legend next. They give a review, then in the end compare it to Uncharted. "BL has some great moments, but overall not as great as Uncharted 2." Ranked #2.

Modern Warfare 2 is next up. "GOTF. I'll be playing this for years." Takes the #1 spot.
Game X follows. "Not as good as MW2. Better than Uncharted 2." Takes the #2 spot.

Now we have:
1 - Call of Duty: Modern Warfare 2
2 - Game X
3 - Uncharted 2
4 - Brutal Legend

Continues from then on. Which game will take down MW2? FIND OUT NEXT EPISODE!

Potential problems:
1. No hard metric. A fast lap in Top Gear gives you a lap time. No arguing there.
2. Same roundtable of reviewers have to conduct the review each time. If guy A reviews game X, then how can guy B rank game Y higher?
3. Fanboy rage guaranteed. Awesome for site hits, high potential for blatant exploitation though.

This may work better for a podcast rather than an actual review site, but yeah that's my idea.
Click to expand...

GameTM (best gaming mag next to Edge) already does this kind of.

They rank every game Worse then/Better then.
 
Durante said:
From the metacritic FAQ: "However, this does pose a problem for our METASCORE computations, which are based on numbers, not qualitative concepts like art and emotions. (If only all of life were like that!) Thus, our staff must assign a numeric score, from 0-100, to each review that is not already scored by the critic."
Click to expand...
Fuck them, seriously.
 
Haunted said:
Fuck them, seriously.
Click to expand...

From the same company that bought you the trusted reviews of Gamespot

"hmm this review reads like a 5 or 6 but it's from our advertisers so... 9.5"
 
For a few years now I've been using Thumbs Up, So-So and Thumbs Down. Which I belive to me a much easier to enterpurtate method of greatness.

Another problem is the reviewer can sometimes be targeting for the wrong audiance when the score is slapped down- problems here as well.
 
ChefRamsay said:
I got a crazy idea. I can't tell if it's good or perhaps the worst review idea ever. That's how crazy it is.

I propose a Top Gear rating system.

Starting TODAY, games are ranked respective to each other. If a site starts with Uncharted 2 for example, it would be rated 1st place, because there are no other games ranked yet.
Brutal Legend next. They give a review, then in the end compare it to Uncharted. "BL has some great moments, but overall not as great as Uncharted 2." Ranked #2.

Modern Warfare 2 is next up. "GOTF. I'll be playing this for years." Takes the #1 spot.
Game X follows. "Not as good as MW2. Better than Uncharted 2." Takes the #2 spot.

Now we have:
1 - Call of Duty: Modern Warfare 2
2 - Game X
3 - Uncharted 2
4 - Brutal Legend

Continues from then on. Which game will take down MW2? FIND OUT NEXT EPISODE!

Potential problems:
1. No hard metric. A fast lap in Top Gear gives you a lap time. No arguing there.
2. Same roundtable of reviewers have to conduct the review each time. If guy A reviews game X, then how can guy B rank game Y higher?
3. Fanboy rage guaranteed. Awesome for site hits, high potential for blatant exploitation though.

This may work better for a podcast rather than an actual review site, but yeah that's my idea.
Click to expand...
make a review site immediately.
 
Haunted said:
See, that's bullshit, because we already have that right now, like on Eurogamer, or Gamespy.

1-6 = terrible
7 = average
8 = decent
9 = good
10= really fucking good

And it just doesn't cut it.



But the current scale is fine? I don't get it.

The problem probably isn't the metric itself, but that aggregation sites exist and convert all the different review metrics - this just doesn't work. :/
Click to expand...

Yeah, you're right. I don't know how to fix it. I'm just gonna go back and fart in my room.
 
doomed1 said:
5 star system. flat numbers, no decimals. provide a point of reference for quality of the game without being too specific.
Click to expand...
***** = Recommended with out hesitation
**** = Recommended strongly, but be aware that it's not perfect.
*** = Some flaws, but overall still good.
** = Rather flawed.
* = Super Death.
 
The absolute first thing that must established is if you are writing your review as a consumer review/purchase guide or if you are actually reviewing the game in a critical sense.

This is in my opinion, where all of these exaggerated 9's and 10's come from. Publications or reviewers want to be able to convey "Go buy this right now" and throwing a high number on it is the easiest way to do it. Meanwhile if you were to look at the game in a critical sense you may agree that it is well worth the money, but the content is not up to par with your publications criteria for those high scores.
 
Ceebs said:
The absolute first thing that must established is if you are writing your review as a consumer review/purchase guide or if you are actually reviewing the game in a critical sense.

This is in my opinion, where all of these exaggerated 9's and 10's come from. Publications or reviewers want to be able to convey "Go buy this right now" and throwing a high number on it is the easiest way to do it. Meanwhile if you were to look at the game in a critical sense you may agree that it is well worth the money, but the content is not up to par with your publications criteria for those high scores.
Click to expand...
A big problem is judging a game based on value.

$60 for 5 AMAZING hours is totally worth it.., and to knock a point off for lenght would be balderdash when to extend it would make it dragged out.
 
Varth said:
I think switching to a simple Buy/Don't Buy/Consider if interested would be a good start.
Click to expand...

If you can't derive a Buy/Don't Buy/Consider message from the copy in current reviews then you're an imbecile.

For those of us whose quality of life doesn't hinge on the metacritic rating of the latest game on our console of choice, the current 'system' is just fine.
 
i'd prefer no metric

like in zero punctuation. he just says what happens in the game, what he likes and dislikes about it, and lets the viewer draw their own conclusions about whether the game is for them or not.

ZP may not be the best example, but you get the point.
 
I agree with an above poster that the current 1-6, 7,8,9,10 system is fine and just like the 5 star system, except when SOME publications try to be "edgy" and mess that up. oh yeah and Metacritic/whatever refuse to convert a 5 star system score into a 6-10 scale.

actually, it's very messed up.
 
My opinion on:

3 Star ratings aka "Thumbs up" "On the Fence" Thumbs down"

Not enough variance for the consumer who wants a quick reference.


10 point scales
Too much baggage with the way reviewers now purposefully use 7 as the average for good.

Decimals in any rating scale.
Absolutely garbage. Creates so much variance the scores become too subjective.


4 and 5 star ratings
Ok but not good enough for my tastes.

Reviews need two parts. The first is the snapshot picture which the score provides but it shouldn't be the only qualifier. A gamer needs to know what type of gamer the reviewer is to begin with. As a result the reviewer should provide a snapshot of their regularly played games and what they consider classics or must haves for their personal collection. The first part tells the reader what the reviewer has done in gaming recently while the the second part tells us what the reviewer has grown up with over the years and deemed to be of the highest quality.
As for the scoring system itself it should be a 7 point scale.
7 - Even if this game type isn't your cup of tea you will be able to respect it for what it is no matter what.
6 - Definitely good. A solid experience that can't be qualified as a top tier title.
5 - There's just enough here to qualify this as something better than mediocre.
4 - Can't make an opinion either way. Lots of pluses and minuses that balance out the experience of this game.
3 - There's enough nagging negative aspects to make this more of disappointing experience.
2 - Definitely bad. Not horrific but still more than just mildly aggravating.
1 - Stay away unless you have a fetish for bad games of this type.


I also really like the OP's idea of mentioning the reviewers play time. All together these 3 items would give a great snapshot unlike just a the current scoring systems.


Of course not only should the snap shot be available but we also need an actual review. I rarely bother with ratings only except as a guideline on which reviews to start reading first. There is a lot to be learned about a game and the reviewers themselves when you start cross-referencing reviews.
 
Rent/Buy/Pass

That's the system I would use.

Or, slightly more granular, I'd break it down into suggestions for fans of the genre and for those that aren't.
 
Some of the suggestions are worse than what they are supposed to replace. For example something similar to Try, Buy, Don't buy is used by Crispy Gamer and it's horrible. Making fewer boxes to fill blurs the distinction between games, leading to black and white comparisons.

The real issue is that good, honest, fair criticism requires discipline, well developed criteria, and a sound philosophy. Some of the best review sites treat the experience as part of a greater conversation, to be tested. It isn't in the process of many sites. A few percentage points difference can be meaningful if there is a good method. Gamespot under Kasavin did a pretty good job being consistent and had the critical mass to follow through with their manifesto:

http://www.gamespot.com/misc/reviewguidelines.html?tag=scoresummary;review-guidelines

As far as I know the above still stands but I doubt the process is in place to maintain its integrity because they've been all over the place in the last years.
 
drakesfortune said:
I don't mind the current system. I think it's fine. I think Buy, Rent, Don't Buy, is far too generic and broad, and doesn't help you identify the really good games from good games. Most people only have time to play the really good games, so I for one am glad for the current ranking system.

I do agree that reviews should be much shorter. Two paragraphs is all I need.
Click to expand...

This is the fault of many reviewers not using a proper composition format. You shouldn't have to read the entire review if you don't want to read multiple pages. The intro, conclusion and maybe the second to last paragraph should cover your needs. A lot of reviews don't do a proper intro.
 
I don't like the idea of rating games based on any kind of economic factor.

I do like a simple four-grade scale as was used by Daily Radar. It forces reviewers to take a stand, to choose whether a game is good or bad and is more representative of how people speak about a game's quality. No one says a game is a seven out of 10 or whatever. They say a game is great, or good, or bad, or crap.
 
I think if we removed numeric values from reviews altogether, there'd be a lot less bullshit drama and it would encourage more people to actually READ reviews.

There is no benefit from putting a number or a letter or emoticon at the end of a review.
 
autobzooty said:
I think if we removed numeric values from reviews altogether, there'd be a lot less bullshit drama and it would encourage more people to actually READ reviews.

There is no benefit from putting a number or a letter or emoticon at the end of a review.
Click to expand...

That SOUNDS mature but there'd be a lot less posts and a lot less threads.

Most of the people here, whether they admit it or not, love that shit.

I mean, check out the Uncharted 2 thread, many peoples' lives hinge on whether it's average review score goes up or down 1 percentage point.
 
I think reviews need two different ratings: one rating for fans of the genre and another for everyone else. I find that there is often a huge disparity between these two ratings. For instance, people will give games like Wii Sports a 6 or worse when to its audience it would be a 9 or so. Also, Fire Emblem games seem to be getting lower and lower ratings because of things that fans of SRPGs likely couldn't care less about. A good version of my system would have a reviewer who likes or loves that genre and a guy who just finds the genre acceptable to rate each game.

This would REALLY help the evaluation of "casual" games to be more correct, and it would also help with niche games.

Oh, and a 10 point system or stars, no decimals, of course.

I also liked the snapshot idea presented earlier, and the better than/worse than idea is interesting.
 
The official metric for reviewing reviews will be based on units from 1 - 100 measuring

Deviation from Press Release

DPR

All 'exclusive' reviews released in any significant period of time before the vast majority of reviews will net an automatic -50 on the DPR rating scale.

Thus has it been spoken. Sit on it.
 
Mindlog said:
The official metric for reviewing reviews will be based on units from 1 - 100 measuring

Deviation from Press Release

DPR

All 'exclusive' reviews released in any significant period of time before the vast majority of reviews will net an automatic -50 on the DPR rating scale.

Thus has it been spoken. Sit on it.
Click to expand...

not sure if i place any validity in "exclusives are unreliable," but this might be a good way to put the press in the right mindset.
 
autobzooty said:
I think if we removed numeric values from reviews altogether, there'd be a lot less bullshit drama and it would encourage more people to actually READ reviews.

There is no benefit from putting a number or a letter or emoticon at the end of a review.
Click to expand...
There are roughly 300 virtual console games. If I decide that I want to buy one, should I read every review first and then decide? Scores serve an important purpose.

I want review scores so I can easily pick the 20 games that are most recommended, read their reviews, and decide for myself which to play.
 
Lyude77 said:
I think reviews need two different ratings: one rating for fans of the genre and another for everyone else. I find that there is often a huge disparity between these two ratings. For instance, people will give games like Wii Sports a 6 or worse when to its audience it would be a 9 or so. Also, Fire Emblem games seem to be getting lower and lower ratings because of things that fans of SRPGs likely couldn't care less about. A good version of my system would have a reviewer who likes or loves that genre and a guy who just finds the genre acceptable to rate each game.
Click to expand...

This is my biggest problem when trying to read reviews from consumers. On sites like Amazon I don't usually see this type of genre hating nonsense but on actual gaming websites like IGN or gamespot this makes filtering the subjectivity an unwanted mental exercise.

the better than/worse than idea is interesting.
Click to expand...

That system is generally good but its most annoying flaw comes up when a reviewer compares games that are from different genres (usually unintentionally because they site the strengths of the game type as flaws when fans of that genre see it differently)
 
a good review needs 3 things
-what kind of game is it?
-what does the critic like about it?
-what doesn't he like?

then I can form my own opinion based on if I agree or disagree with him.


a good review does not need a score, but if it has to have one for the average consumer or illiterate, I think a maxium of 5 (like most here already said) or maybe 10 would be more than enough.
 
Ardorx said:
Here's an idea: Don't apply a number or letter grade to it. Just write your review and let people read it and make the decision from there.
Click to expand...

Yep thats the way to go.

You should write about the basics (gameplay, graphics..) then about the story(without spoilering anything!), compare it to other games (If you like that game you'll probably like this one) and your personal impressions. Not necesseraly in that order..
 
How about just do away with the metric entirely and let the writing stand for itself, crazy concept I know.

edit: like has been said many times already, just in this thread alone
 
If could change one thing about reviews, it'd be to make them shorter. I don't care about the game's concept. I don't care about the developer's history. If I cared about those things, I'd look them up elsewhere. You're not going to beat Wikipedia. Don't review any category you find useless--taking two paragraphs to tell me that the controls aren't notably good or bad is stupid. Taking two paragraphs to tell me that the game runs well but not perfectly is stupid. Taking a page to tell me that the graphics are mostly good but could be better is stupid. Tell me how the game is notable, whether that's in a good way or a bad way. Get rid of all score categories except overall score.

As of right now, the overall score is significantly tilted by categories that have no impact on the person's enjoyment of the game. Even "not an average" reviewers like IGN are kneecapped by scoring each individual category.

Get rid of these things and you'll be able to review 90%+ of games in 250 words, seriously.

Here's Robert Christgau's review of Jay-Z's latest album:
For a record consisting almost entirely of boasts about being the best, the ex-prexy's official comeback -- and also, let it be noted, his inaugural project with or is it for his new corporate partner -- is fairly superb. He brings it off because he is the best, because he's documented more achievements than most bigmouths, and because he holds chits for miles. Not only are chief beatmakers Timbaland and Kanye West co-equals, he's gotten A work out of them -- cf. Timbo's sample-free spirals on the atypically unbraggadocious "Venus Vs. Mars" and the atypically staccato clap-for-'em West designs for "A Star Is Born." Both are buried mid-disc, just where you'd think Jay would be sneaking in the weak bleep. None of that here -- though you have a right to think he's coming on too strong.
Click to expand...

Christgau does, basically, 100-200 word reviews of the biggest albums. Below those albums, he has "Honourable Mention" albums, for which he'll do a 1-2 sentence blurb. Below that, he has a category called "Choice Cuts" where he just lists a song or two that stand out on an otherwise unremarkable album. At the bottom, he has "Dud of the Month", an album bad enough to require 100-200 words.

Not saying that Christgau is infalliable; there's plenty to critique in his work, but there's something to be said for writing dense, short reviews, and not using words to explain anything that doesn't require explanation. A game that's perfectly average in every single way deserves a single word review--"meh."

Unfortunately the more concise reviews in the gaming industry aren't any good because they just take the conventional review and lose half the word count. They don't actually adjust the content to stop covering the stuff that's not notable.
 
Different scales for different genres.

Example, for MMO you have

Gameplay

Social

Repetivness

etc etc.

For FPS you have:

Controlls

AI

etc etc.
 
I'd like a meh rating system.

The more mehs a game gets, the shittier it is.

If it gets no mehs at all, it's alright.
 
Lyude77 said:
I think reviews need two different ratings: one rating for fans of the genre and another for everyone else. I find that there is often a huge disparity between these two ratings. For instance, people will give games like Wii Sports a 6 or worse when to its audience it would be a 9 or so. Also, Fire Emblem games seem to be getting lower and lower ratings because of things that fans of SRPGs likely couldn't care less about. A good version of my system would have a reviewer who likes or loves that genre and a guy who just finds the genre acceptable to rate each game.
Click to expand...
The problem here is that the solutions to the problems often create more problems themselves. Unless the publisher is just 100% comfortable with the niche they've carved, with each iteration they should be trying to not only satisfy the existing fans, but also trying to expand the audience. Your mentioning of Fire Emblem is a good example of what I'm talking about. Dynasty Warriors would probably be another one (I'm not suggesting that their quality is comparable), or you could go back to the NES days and mention something like Mega Man.

Sending in your SRPG superfan isn't going to be useful to the people who gradually became disinterested in the franchise over time, or were initially not drawn to the genre. For these people, sending in the fan to declare "Nintendo does it again and serves up another solid outing!" really doesn't mean anything.

But I do understand your point all the same.
 
It won't happen because games aren't art and therefore are judged on technical merits. For example, Graphics - Sounds - Lasting Value from IGN's criteria are technical descriptors with smaller amounts of room for subjectivity. Therefore any game that looks good, sounds good and has length will pretty well even if its completely uninspired.

With reviews of art, like movies, they are graded solely on the subjective experience of the reviewer. The difference will always persist so you'll never see a 5 star rating system so commonplace in film.

Technical evaluations can be very precise. Cars and electronics are often graded this way because intricate comparisons are important to consumers in those fields. Giving a game an 88 is has more in common with giving Anti Virus software 88 than Transformers a two stars.
 
You people who don't like percentiles and 100 point systems must have hated school. Yes there is a need for 100 different variations, the difference between an 88 and an 82 is little, still a B, but just like when a professor grades your paper, it's not merely outstanding, pass or fail.

There's a lot of things to consider, and it could be an 82 instead of an 88 for something like the oversight of a feature that feels like it should have been in but really wouldn't have altered the gameplay at all. Like the lack of a jump in brutal legend for example.

And a 5 star system can, and is (according to places like Gamerankings and Metacritic) easily extrapolated to a 100 point system.

If a game you like gets a 3/5 or a 60% it doesn't matter. If instead of 60% it gets 63% well you know it did just a little bit more right than a game that did get 60%. At least with the 100 point system they can differentiate it.

I am a big fan of Direct Hit, Hit, Miss and Dud though. That Daily Radar used to use. Destructoid's Buy It, Rent it or Skip It works too. But something about Direct Hit and Dud was awesome. Still it was extrapolated into the 100 point scale anyway so it really didn't matter.
 
The problem with review scores aren't the metric but how both reviewers and readers take them way too seriously. They're ultimately one dude's opinion on a game. However, people see them as a purely objective statement about the overall quality of the product. Since video games are for entertainment, and aren't some sort of utility, that's impossible. Reviewers fall into this problem as well, trying to balance their reviews by how other people would like it.

Take a look at a recent, well received game on Metacritic. You'll see almost all good reviews, 90% and up. Take a look at a recent, well received movie on Rottentomatoes. You'll see the whole range from people hating it to loving it. Likewise, I'm sure almost everyone has a game they loved that reviewers didn't, or hated that reviewers loved.

If reviewers and readers just accepted the fact that every single review is just one guy's opinion, and for any real information about the game you'd have to read the review, things would work out better for everyone.
 
A simple play, don't play would be good enough for me. The only thing I'd add is maybe have one text review and then a bunch of reviewers with their play or don't play.

I'm playing Cryostasis right now and it's the perfect example of why a 10 point scale or A-F scores don't work. The combat is clunky and the game chugs along, and if you read the reviews they all say don't play because of these two reasons and give the game a 6 or so. If you're doing a "subtract from 10" review where you just point out the flaws, you're going to come up with a 6, but you'd be missing an incredibly unique experience if you skipped Cryostasis. Compare it to other first person games - I'd easily recommend Cryostasis with its flaws over a standard competent FPS like FEAR2.
 
You must log in or register to reply here.

Similar threads

Big Hops, a new 3D platformer that is getting good reviews
2
131
Gamer_By_Proxy replied
MOUSE: P.I. For Hire - Reviews Thread
2 3
105
961
vodka-bull replied
IGN reviews the "new" Commodore 64
13
534
IbizaPocholo replied
Pragmata - Reviews Thread
3 4 5 6 7
343
1K
Fabieter replied
Marathon - Reviews Thread
3 4 5 6 7
315
3K
Bernardougf replied
Top Bottom