It's a metric, yeah. But it doesn't lead to any of the problems I tried to point out in this article. And it actually means something - when you want to know whether a game is worth your time, "8.6/10" isn't very helpful.Last time I checked, Kotaku used a three point scoring scale: "Yes", "Not Yet" and "No" (in descending order).
p)
-
Hey Guest. Check out your NeoGAF Wrapped 2025 results here!
Kotaku: The Problem With Review Scores
- Thread starter JABEE
- Start date
MaximoffZero
Banned
Rating a person's opinion, on a game or otherwise, using a numeric scale is pointless. Opinions are not quantifiable. Period.
However, I know that there are people that just want to get to the point and not have to read review after review. I think that can be done without a score though.
There was a gaming website I used to visit back in the dial-up days (can't remember the name now) that would have a section at the bottom of the review literally titled "bottom line" where they would sum up their opinion in a single sentence. It was a great way to get the idea when I was in a hurry.
And by the way, the best "bottom line" I read on that site was for Tekken 3. All it said was "buy it."
However, I know that there are people that just want to get to the point and not have to read review after review. I think that can be done without a score though.
There was a gaming website I used to visit back in the dial-up days (can't remember the name now) that would have a section at the bottom of the review literally titled "bottom line" where they would sum up their opinion in a single sentence. It was a great way to get the idea when I was in a hurry.
And by the way, the best "bottom line" I read on that site was for Tekken 3. All it said was "buy it."
Vanguardofyourdestruction
Banned
Gotta cater to what the masses want. Most gamers won't bother reading a review, rather just check the score and see how it compares to other games.
I disagree, I like review scores.
However a lot of journalists make me wonder about their honesty. Rome 2 for example launched with some heavy issues but got great review scores anyway. Skyrim has some massive launch issues but top scores all around. If an indie game launched like that it'd just be shot down. Why do AAA games get away with it?
So far I've found Rev3 games the best and their video reviews seem honest.
However a lot of journalists make me wonder about their honesty. Rome 2 for example launched with some heavy issues but got great review scores anyway. Skyrim has some massive launch issues but top scores all around. If an indie game launched like that it'd just be shot down. Why do AAA games get away with it?
So far I've found Rev3 games the best and their video reviews seem honest.
oconnomiyaki
Member
Mona Lisa. 8/10.
They're stupid, and the only reasons to justify them are the people who lack basic comprehension skills -- "I can't tell if I should go see it... give it a number please." -- and sites like Metacritc that require the lie of objective, discrete quantifiers to exist.
I'd go so far as to say reviews are stupid. They're just the opinions of a person and a reflection of their own personal tastes. Why should I care about the opinions of many of these professionals over my mates or even the guys at the supermarket who like to chew the fat about games while scanning my groceries? Because there's an English degree and a formal soapbox there? Cool, I had one of those two and it means shit.
I found Uncharted 3 to be an unenjoyable slog filled with frustrating set pieces and that goddamn dune sequence. The sandstorm shootout in the end would have made me quit if it wasn't so obvious I was near the end.
So why bother with reviews at all if they're just going to be someone saying whether they like the game? That means fuck all to me.
They're stupid, and the only reasons to justify them are the people who lack basic comprehension skills -- "I can't tell if I should go see it... give it a number please." -- and sites like Metacritc that require the lie of objective, discrete quantifiers to exist.
I'd go so far as to say reviews are stupid. They're just the opinions of a person and a reflection of their own personal tastes. Why should I care about the opinions of many of these professionals over my mates or even the guys at the supermarket who like to chew the fat about games while scanning my groceries? Because there's an English degree and a formal soapbox there? Cool, I had one of those two and it means shit.
I found Uncharted 3 to be an unenjoyable slog filled with frustrating set pieces and that goddamn dune sequence. The sandstorm shootout in the end would have made me quit if it wasn't so obvious I was near the end.
So why bother with reviews at all if they're just going to be someone saying whether they like the game? That means fuck all to me.
Karak
Member
That is really good!
I am currently using buy, wait for a sale, rent, and stay away from. But I like yours a lot.
FriedConsole
Banned
No the law of averages says that if out of 100 reviews and 90 of them are positive, chances are it is a well made game and if I like that genre of game I will probably like it and is worth taking my chance with money. There is no perfect formula that says I will absolutely like a game but I can increase my chances by looking at aggregate scores. I could probably do the same by reading 100 reviews but then I would be spending 10 hours on my purchase time that I rather spend playing a game. If I just read one or two reviews, how do I know I just didn't pick out reviewers that are total assholes and then I got a wrong opinion on a game?
Theoretically something like a 10 point scale also means something. The problem is not merely the inability of a solitary number to convey enough information, but also a reader's lack of desire to read both the review and the way the site's review scale operates, which combined with a tendency to treat all review scales as equivalent leads to scenarios where people get outraged over numbers they don't really understand. It also doesn't help that people treat reviews as if they are supposed to somehow objectively quantify and measure the excellence and quality of a game while entirely removed from any kind of human biases and subjectivity.
RobFox64tm
Member
I agree one hundred percent. It's unfortunate how important of a marketing tool they've become for publishers though.
I actually tend to think that usually means if I'm really interested in it then it's almost certainly worth grabbing, but part of the problem IS how people have been reacting to scores. Even if the review's positive and just points out the game has some issues that don't ruin the game some will act like an 8's a disaster if it's for some big release despite being a very good score. Nevermind those who don't actually HAVE issues to bring up and just don't feel a game's THAT great and deserves a 9 or 10, treating reviews as if they're some schoolwork that if you don't get any wrong answers you'll get a perfect score, even though you wouldn't treat a creative product the same as a math test. Hell, I don't think you can treat many personal evaluations the same as that ANYWAY, we're operating on a basis of "will I enjoy this/will this work great for me" and not "did you manage to not screw up?" as we've undoubtedly all played game that didn't really do anything WRONG, but we just didn't enjoy.
This is huge. So many games, SimCity, Diablo 3, GTA 4 for example were loved to death out of the gate but as they digested are universally seen as meh at best. I'm not even talking about the connection issues with Simcity but the game itself was realized to be shallow.
I agree. Review scores were born as a marketing decision to lend authority and definitveness to the opinions of an outlet's critic.
Many scoring systems now exist as an attempt to make it as easy as possible for a PR person to justify sending an outlet an early copy of a game.
dancingphlower
Member
Kotaku's Yes/No system is not without flaws either. Remember the Dragon's Dogma incident? Adding that big red "NO" to the review made most of the discussion about that "NO" rather than about the actual review.
I used to give a shit back in the 90s, and only purchased games that were 8 and above. Now it just doesn't matter much to me, and am playing great games that I had missed because of my own stupidity. All these younguns get into arguments over it like it was the SNES/Genesis playground wars.
Using the width of a game's appeal as a designation of quality seems like a truly horrible way of going about it. I don't think it is the way to design a truly fantastic game and I don't think it is the way to review one.
oconnomiyaki
Member
Second-guessing your audience is a terrible idea. The best you can do is justify why you thought a game was good and let the reader decide, based on your provided evidence, whether you're a muppet or not.
hey_it's_that_dog
benevolent sexism
The numbers are in no way arbitrary or meaningless, but I agree that they are toxic to discussion. That's the fault of the discussants, however, and not the review system.
AcademicSaucer
Member
Wait for pokemon x/y reviews .
Hot Coldman
Banned
The problem with scores is that even the best writers have a tendency to be really fucking bad at picking a score that actually fits what they've just written. I'm not talking about "oh you gave this game 4/5 but only spoke about the negatives!" because whatever, that's a chosen style; I'm more on about writers that will hand out a perfect score and just deliver the driest fucking review.
I like EDGE, because when they hand out the elusive 10 (well, not so elusive nowadays but w/e) you believe it. Hell, in that instance the score is useful because, as someone who rarely reads reviews, I'll always read an EDGE 10. Because there's certainty that it'll be worth the read.
I like EDGE, because when they hand out the elusive 10 (well, not so elusive nowadays but w/e) you believe it. Hell, in that instance the score is useful because, as someone who rarely reads reviews, I'll always read an EDGE 10. Because there's certainty that it'll be worth the read.
Review scores can never match the text. How can a number convey the information you're looking for?
Take the Escapist Review of GTAV. 3.5/5 stars, the reviewer explained certain details he didn't like. Other people like those details though, you can decide that this is something for you based on the text.
A number doesn't tell you a single thing except how a single person felt about a game at a particular point in time. The actual review is usually where there's some sort of useful information for you to make your purchasing decision.
Well, I made a comment in the GTA V thread about this. I was half-joking, basically taking a comment literally regardless of the context. However, reading this article makes that point somewhat relevant.
Obviously the main purpose of a review score is to help the public decide whether a product is worth purchasing. You could completely do away with a numerical score system, but by saying a game is excellent, good, bad, or average, it would still be a metric in which the quality of a game is judged. Just because some gamers obsess over scores in a negative manner, it doesn't necessarily mean they are without worth. Saying a game is 'average' when other sites have claimed it is 'good' would still probably get the same negative reaction as it would if you simply assigned a number to the score.
That said, I've always thought a '5 out of 5' score system is better than '10 out of 10'.
Obviously the main purpose of a review score is to help the public decide whether a product is worth purchasing. You could completely do away with a numerical score system, but by saying a game is excellent, good, bad, or average, it would still be a metric in which the quality of a game is judged. Just because some gamers obsess over scores in a negative manner, it doesn't necessarily mean they are without worth. Saying a game is 'average' when other sites have claimed it is 'good' would still probably get the same negative reaction as it would if you simply assigned a number to the score.
That said, I've always thought a '5 out of 5' score system is better than '10 out of 10'.
Bringer Of Bacon
Member
Agreed
One thing to keep in mind is that review scores being problematic is pretty much unique to video games as a medium. The fact that there is much more variance in review scores for both films and albums, and music as well, demonstrates one part of the problem. We have discussed how broken the use of the 10 point scale has been for years now, and that simply isn't the case with other media.
Like some have already mentioned in this thread, review scores in and of themselves are not bad. What is toxic is the way that gamers tend to discuss them amongst themselves. It's like they can't decide whether or not they actually have a problem with the abuse of the 10 point scale or simply have a problem when reviewers disagree with their opinion. Whichever it is, it reflects more about the gaming community than it does review scores in general.
For probably a plethora of reasons, gaming enthusiasts seem to be much more entitled and inflammatory than the fanbases of other popular mediums.
Like some have already mentioned in this thread, review scores in and of themselves are not bad. What is toxic is the way that gamers tend to discuss them amongst themselves. It's like they can't decide whether or not they actually have a problem with the abuse of the 10 point scale or simply have a problem when reviewers disagree with their opinion. Whichever it is, it reflects more about the gaming community than it does review scores in general.
For probably a plethora of reasons, gaming enthusiasts seem to be much more entitled and inflammatory than the fanbases of other popular mediums.
Does a score tell you that the game increasingly gets difficult as you get to the end? Does a score tell you that the game has a wonderful story but flawed mechanics? Does a score tell you that this game has a 40 hour campaign and 20 hours of side content??
An arbitrary number is NO WAY going to be able to tell you if a game is worth your "time and money". You have to read the the text and learn about each individual games nuances in order to compare them. Most importantly, you have to PLAY them. But you know this already....
I've played hundreds of 5-7 scored games that completely outshine 8-10 scoring games... Its about the experience and the overall package... Not the number some reviewer dictates to it...
hey_it's_that_dog
benevolent sexism
Hey does anyone in here actually know what "arbitrary" means?
Kotaku's review system is my favorite of every site out there because it doesn't bother with numbers except the objective ones like fps, resolution, and the amount of time it took the reviewer to complete (which obviously will be different depending on who's playing, but is still somewhat helpful to know)
Scores are just fanboy fodder that make people think there's some level of objectivity in an inherently subjective review.
Scores are just fanboy fodder that make people think there's some level of objectivity in an inherently subjective review.
based on random choice or personal whim, rather than any reason or system.
Most of the time , that's EXACTLY what a game review score is
Haunted
Member
Impossible to do for the mainstream sites in an environment where exclusives and "being first" to get clicks is more important than journalistic integrity and the quality of coverage.
Several weeks of decompression period means no one cares about the review when it hits eventually, because publisher X has released hyped game Y in the meantime and all the advertisement focuses on that title instead.
It's all about priorities in videogame journalism, and netting hits and clicks is more important than a proper critical process, just like keeping a positive relationship with
edit: this is not a condemnation of journalists, but a criticism of the system they have to enter and conform to on the way to that community manager job
I'm curious as to how you think the two are meaningfully different.
Last time I checked, Kotaku used a three point scoring scale: "Yes", "Not Yet" and "No" (in descending order).
Actually, while that score system is interesting, I think it is a little vague. A 'not yet' score would leave me a little confused. Maybe reading the full review would give me a clearer picture, but then again, maybe not.
I really like their reviews they state what they liked disliked , what they played and if they recommend it .A system i wish was mass adopted
Less gradient leaves the score as a little bit more of a general impression. So you end up with a scale where there is no score difference between a great game and a "genre defining" experience and in so doing helps limit some of the hyperbolic exaggerations that can be hard to avoid in the heat of the moment.
5 point scale is the best scale.
hey_it's_that_dog
benevolent sexism
So you are claiming in complete seriousness that the score is in no way correlated with how much the reviewer enjoyed the game?
Scores are imprecise, and that bothers some people to an extreme degree, but arbitrary they are not.
The entire field of social science is built on quantifying things that are difficult to quantify precisely. Reviewing games by assigning a number on a scale to a subjective experience is not very different from that. Again, a number can only capture someone's experience at a gross level, and this is why numbers are paired with paragraphs filled with words. But the numbers are not random or based purely on a whim.
If scores are problematic to discussion, it's because the people having the discussion are too fucking stupid to understand what these numbers really mean and how seriously they should be taken. (i.e., Somewhat seriously as a general indicator, but not useful in comparing two games that differ by 3 points in their metacritic average.) I know people need the rigidity and supposed objectivity that numbers provide so they can cite them while wasting their time arguing about dumb shit on the internet, but it would be great if everyone stopped doing that, but also if everyone else stopped overreacting and claiming numbers are literally meaningless.
The YES/NO is a waste of time for me because it's probably using a different threshold than I am. If I have all the free time in the world, I'll probably see everything 7.0+ as a YES. If I only have time for a couple of games a year, I'm probably waiting for a 9.5+ to give it a YES.
It's not a matter of if something is worthwhile, but how worthwhile it is. Had The Last of Us not been stellar, I probably would have waited some time to get on board, maybe even waiting for the eventual PS+ release. Since it got all tens, I knew it would be a topic of discussion amongst people and I wanted to be able to take part in that, so I picked the game up and played through it straight away.
Additionally, scores can prime a review in advantageous ways. For your average game, I probably have no great interest in actually reading the review. Describing mediocrity is often itself mediocre. So, if I see an unusually high or low score, I'm much more interested in reading the review to find out just how it arrived at that perception.
I won't pretend for a moment that review scores are the be-all/end-all, or that they are without flaws, but they remain important and integral to the industry.
edit: and another thing: it doesn't bother me in the slightest that reviews are top heavy, mostly in the 7-10 spectrum. Most games are pretty decent, compared to what they could be. Most developers are at least somewhat competent, and most publishers know when to drop a project that isn't going to be any good. Complaining about top-heavy scores is like complaining about a lack of short people playing professional basketball - you don't get to become professional by being terrible.
It's not a matter of if something is worthwhile, but how worthwhile it is. Had The Last of Us not been stellar, I probably would have waited some time to get on board, maybe even waiting for the eventual PS+ release. Since it got all tens, I knew it would be a topic of discussion amongst people and I wanted to be able to take part in that, so I picked the game up and played through it straight away.
Additionally, scores can prime a review in advantageous ways. For your average game, I probably have no great interest in actually reading the review. Describing mediocrity is often itself mediocre. So, if I see an unusually high or low score, I'm much more interested in reading the review to find out just how it arrived at that perception.
I won't pretend for a moment that review scores are the be-all/end-all, or that they are without flaws, but they remain important and integral to the industry.
edit: and another thing: it doesn't bother me in the slightest that reviews are top heavy, mostly in the 7-10 spectrum. Most games are pretty decent, compared to what they could be. Most developers are at least somewhat competent, and most publishers know when to drop a project that isn't going to be any good. Complaining about top-heavy scores is like complaining about a lack of short people playing professional basketball - you don't get to become professional by being terrible.
That essentially boils down as scores under another name. The point of scores hurting reviews is that they are often the focus of discussion instead of the actual review.
Using those parameters wouldn't really be that different from using scores.
I think the industry as a whole cares too much about review scores but that they aren't bad in and of themselves. Reviews can be very useful in judging whether this game meets my quality standards or suits my tastes as a gamer. But to fixate over a number is crazy. Especially when people won't buy games below an 8 score or publishers will give bonuses based on review scores.
But review scores in the form of a grade (A+,B-) or stars out of 4, or a number out of 10 is still quite common in the film industry. It just doesn't seem to have as direct an effect on ticket sales or consumer interest. I don't why videogame buyers care so much about review scores except perhaps because games cost significantly more than a movie ticket.
But review scores in the form of a grade (A+,B-) or stars out of 4, or a number out of 10 is still quite common in the film industry. It just doesn't seem to have as direct an effect on ticket sales or consumer interest. I don't why videogame buyers care so much about review scores except perhaps because games cost significantly more than a movie ticket.
Femmeworth
Banned
Scores wouldn't be so bad if the reviewers/websites actually had a methodology.
Well, it is less nuanced, but still providing a good metric to formulate an idea of the quality of a product.
It is more about how you assign meaning to those numbers.
1 = Awful
2 = Bad
3= Average
4 = Good
5 = Excellent
You still get a clear picture of the quality of a product while at the same time doing away with pointless arbitrary numbers. What exactly makes a game 7.5 rather than an 8 for instance? What does that mean in terms of the quality of a product?
I also like that Kotaku doesn't assign games a number, but I suspect it is going to be tough for other sites to follow suite because most websites don't get the kind of traffic that they do.
The scores are definitely part of the problem at least. How often does a AAA game get scores lower than 7? You'll see maybe 1 or 2 out of like 50+ reviews for most of them.
Just checking AC3 for a minute and it has 3 scores lower than 7 out of 61 reviews (84 average) and they were all 6-6.5. That's for a game a good majority of us would agree is really bad in most areas.
There's a huge problem with how the media is able to critique games. Eliminating scoring wouldn't solve much right away but it would at least force people to actually read the damn things and maybe then they would start seeing just how inept most media people are at judging games. THEN maybe things would start changing.