• Hey Guest. Check out your NeoGAF Wrapped 2025 results here!

I decided to create a new, better review aggregator

Status
Not open for further replies.
Reviewers who refuse to give games a 10 are like teachers who refuse to give out the best grade on principle

Wrong. They are wrong.
 
So where did the bad review touch you? Does it really matter? A good game is good regardless of its review score. A bad game is bad regardless of its review score. Zelda being 97 imstead of 98 doesn't mean anything nor does HZD being 89 instead of 90 mean anything. If someone was on the fence a point really would not make a difference. However OP, you are missing the point. An aggregate is for everything not just what you like.
 
Onyar said:
when you use an scale of 10, yes?
can you elaborate?
Click to expand...

It was elaborated dozens of times in different threads already, many magazines also have a chart or section on their site telling you what their scores mean.

Let's quote IGN for example:

10.0 - MASTERPIECE

Simply put: this is our highest recommendation. There's no such thing as a truly perfect game, but those that earn a Masterpiece label from IGN come as close as we could reasonably hope for. These are classics in the making that we hope and expect will influence game design for years to come, as other developers learn from their shining examples.
Click to expand...

Zelda for example was titled a masterpiece quite often but I've never read any review that called it perfect, there's a line between being perfect and exceptional.
 
I like how 8 is already going to red/orange because it is bad. /s

Come on. It would be repetitive what I have to say, but just to make sure...
Too little major-well-trusted-gaming-websites and who's to say which gaming review website is trusted?

A review is something that is very, very much relative and subjective.
If it's not a technical review (framerate consistency, rendering, etc) there are almost no rule to be follow.

If you really, really wish to develop a new aggregator for reviews, and you like the idea of looking for some reviewers only, you can create the "bubble effect" for your system.

Bubble Effect
List the reviewers you like the most to hear about, their review will be the ones displayed to you and you will even see the games you like rated only by their review.
Other reviews would be hidden with a "show more", if needed.

It's not a sarcastic suggestion, as it may sound, but, if you really want this idea I think this would be a better approach.
 
Z5nrtwL.png
 
ViciousDS said:
So how many of those major outlets you have listed gave Zelda a perfect score?
Click to expand...

It's to be expected, Zelda amassed 54 "perfect" scores out of 86 ratings.

Oh there we have our final description what a 10 is, it's a perfect score as there cannot be a higher one, however it doesn't mean the game is perfect.
 
There's no value to excluding sites from a review aggregator once they pass a level of notability. Excluding outliers is counter-productive, too - outlets are largely homogeneous, with little variation in opinions offered. Why make that worse? At least the guy scoring Breath of the Wild 5 out of 10 might have something different to say.
 
If you want to see a 100% for Zelda, just treat metacritic as if it were rottentomatoes. Not even the three mixed reviews scored it low enough to prevent a 100% fresh rating ;P
 
Onyar said:
Any person who give a 10/10 a game simply isn't trustable as a review if it tries to be objective.

And that's because 10/10 means perfection, the best at the moment for everyone.
The reviewer has to think all the cons and pros that everybody could find, for example for Zelda you have to think about people who doesn't enjoy the graphics, or for the Dark Souls for the people who doesn't want to fight the mechanics to enjoy the game. The best of all is that these are just opinions, but opinions that some people could find.

Said that in my opinion the best way to review a game is compare with others games and themes, like: if you enjoy CRPGs, liked Divinity: OS and enjoy fantasy stories, buy it. If someting of these doesn't appeal you gave it a try if you have the money and the time, if you didn't like any of these, just don't play it.
Click to expand...
That's not what reviews are. Reviews are a subjective appraisal of an item and also 10/10 does not mean perfect, because reviews and scoring is, once again, subjective.
 
Onyar said:
Any person who give a 10/10 a game simply isn't trustable as a review if it tries to be objective.

And that's because 10/10 means perfection, the best at the moment for everyone.
The reviewer has to think all the cons and pros that everybody could find, for example for Zelda you have to think about people who doesn't enjoy the graphics, or for the Dark Souls for the people who doesn't want to fight the mechanics to enjoy the game. The best of all is that these are just opinions, but opinions that some people could find.

Said that in my opinion the best way to review a game is compare with others games and themes, like: if you enjoy CRPGs, liked Divinity: OS and enjoy fantasy stories, buy it. If someting of these doesn't appeal you gave it a try if you have the money and the time, if you didn't like any of these, just don't play it.
Click to expand...

That is a ridiculous stance. I don't think most reviewers, if any, define 10/10 to be a perfect game. They usually define it as their highest level of recommendation. It means that despite the flaws (which all games have) the game is still very enjoyable and made with a high level of quality. On top of that most reviewers have their own personal criteria for giving perfect scores like how innovative it was or how bug free it was or whether it runs at 60 fps etc. Don't treat review scores to be an exam result or an objective measurement. It's neither of those things.
 
Sho_Nuff82 said:
Did you read that link?
Click to expand...

Yeah? If your point is the "63% higher than the average critic" part, that's a fairly meaningless statistic imo. If an outlet rated every single game 1 point higher than the average, they would have 100% of their games in the green without exerting any significant force on the metascore.
 
Robotguy said:
Maybe he should create an AI review aggregator that reads the reviews and aggregates the text.
Click to expand...

I came into this thread expecting a Rotten Tomatoes-style system (which would kinda be interesting to see for games) but this idea right here instantly gained 100% of my support, it'd be either fascinating or hilarious or both.
 
The problem with aggregators is that reviewers don't use a standard rubric or judging criteria. Even if you remove "certain" outlets, there's no standard method of review.

Aggregators will always have this problem. Creating your own with selective reporting does little to solve the issue.
 
I can see an argument for improving the system metacritic uses. I cannot in any way support such a small sample size, however. That's the opposite of what aggregate sites are for. May as well slap an 8.5 on every major AAA release and call it a day.

I personally like to read a few really high review scores, a few really low ones, and a few in the middle. Aggregate sites serve a purpose (even if it's not one that's essential) but there's no sense in getting riled up because the score bothers you. Someone's opinion isn't invalid because they gave your favorite Zelda a 7, OP.
 
Rinj von Scoot said:
I came into this thread expecting a Rotten Tomatoes-style system (which would kinda be interesting to see for games) but this idea right here instantly gained 100% of my support, it'd be either fascinating or hilarious or both.
Click to expand...
Opencritic has one. It's shit because games skew so high on the review score scale.
 
N.Domixis said:
What should happen, all reviews are allowed from the beginning, as soon as a a reviewer scores a 7 or below if the game is at 90+ they are banned. If a 60 or below at 80+, banned, etc. etc. Aditionally, for every 100 a game scores we are allowed to drop two of the lowest scores. If the score is 20 away from the average it's removed. All this should filter out the trolls from the authentic reviews.
Click to expand...
What exactly makes an outlier review a troll review that has to be cut from the authentic whole?
 
MTC100 said:
It was elaborated dozens of times in different threads already, many magazines also have a chart or section on their site telling you what their scores mean.

Let's quote IGN for example:



Zelda for example was titled a masterpiece quite often but I've never read any review that called it perfect, there's a line between being perfect and exceptional.
Click to expand...

It's great that they elaborate something the score, not everybody does.
But then we aren't reviewing how the game is, we review if this game is good to be played now. Then we should divide how actually the game is (mechanics, grafics, etc) to how I enjoyed (10/10 I enjoyed a lot). But as first thing is more objective and the second is more subjective they tend to don't do it.

What IGN said is that everybody actually find it very funny to play, but that's not reviewing a game it's reviewing an experience.
 
If I was making a review aggregator, I'd use the median review score rather than the mean. That way outliers wouldn't have an outsized impact on the score.
 
You'll never get a more fair result by excluding sites left and right. Perhaps a system where the top 15% and bottom 15% review are ignored for the average score would work better for excluding the anomalies and perhaps biased reviews, but by making a system with less reviewers you're effectively creating a more unfair and worse system.
 
Onyar said:
It's great that they elaborate something the score, not everybody does.
But then we aren't reviewing how the game is, we review if this game is good to be played now. Then we should divide how actually the game is (mechanics, grafics, etc) to how I enjoyed (10/10 I enjoyed a lot). But as first thing is more objective and the second is more subjective they tend to don't do it.

What IGN said is that everybody actually find it very funny to play, but that's not reviewing a game it's reviewing an experience.
Click to expand...

You can't objectively review mechanics, grafix, etc. You can subjectively review your experience with these various things though.
 
N.Domixis said:
What should happen, all reviews are allowed from the beginning, as soon as a a reviewer scores a 7 or below if the game is at 90+ they are banned. If a 60 or below at 80+, banned, etc. etc. Aditionally, for every 100 a game scores we are allowed to drop two of the lowest scores. If the score is 20 away from the average it's removed. All this should filter out the trolls from the authentic reviews.
Click to expand...

lmao,

I really can't tell who is serious and who is joking anymore.
 
Stumpokapow said:
From a conceptual standpoint, I think this is like OpenCritic but strictly worse. But also I don't think OpenCritic has done a good enough job criticizing Metacritic's use of weighting.
Click to expand...

I think the worst part is how bad we've done at showing you can customize your outlet selection...

The whole point of OpenCritic is that you can pick and choose which outlets you want in your score.

Metacritic uses editorial judgment over which outlets are included, and to what degree they're included (ie weighting).

OpenCritic believes that review aggregators shouldn't require editorial judgment. Period.

If the OP wants to pick-and-choose his outlets, he can do that today on OpenCritic:

6bjcKBw.png
 
MrPanic said:
You'll never get a more fair result by excluding sites left and right. Perhaps a system where the top 15% and bottom 15% review are ignored for the average score would work better for excluding the anomalies and perhaps biased reviews, but by making a system with less reviewers you're effectively creating a more unfair and worse system.
Click to expand...
Pretty much. Ignoring the extreme outliers doesn't sound like a bad idea, but the reviewer pool should remain big.
I suspect if you just take the most prominent mainstream publications you end up with fairly similar scores anyway.
 
Sigh.

Onyar said:
Any person who give a 10/10 a game simply isn't trustable as a review if it tries to be objective.

And that's because 10/10 means perfection
Click to expand...

SIGH.

N.Domixis said:
What should happen, all reviews are allowed from the beginning, as soon as a a reviewer scores a 7 or below if the game is at 90+ they are banned. If a 60 or below at 80+, banned, etc. etc. Aditionally, for every 100 a game scores we are allowed to drop two of the lowest scores. If the score is 20 away from the average it's removed. All this should filter out the trolls from the authentic reviews.
Click to expand...

SIGHHHHHHHHHH.
 
Onyar said:
It's great that they elaborate something the score, not everybody does.
But then we aren't reviewing how the game is, we review if this game is good to be played now. Then we should divide how actually the game is (mechanics, grafics, etc) to how I enjoyed (10/10 I enjoyed a lot). But as first thing is more objective and the second is more subjective they tend to don't do it.

What IGN said is that everybody actually find it very funny to play, but that's not reviewing a game it's reviewing an experience.
Click to expand...

Everyone who writes a review elaborates the score. The written content of the review is doing exactly that.

Also please tell me how do you objectively measure graphics and gameplay.
 
SomedayTheFire said:
That's not what reviews are. Reviews are a subjective appraisal of an item and also 10/10 does not mean perfect, because reviews and scoring is, once again, subjective.
Click to expand...

Arion said:
That is a ridiculous stance. I don't think most reviewers, if any, define 10/10 to be a perfect game. They usually define it as their highest level of recommendation. It means that despite the flaws (which all games have) the game is still very enjoyable and made with a high level of quality. On top of that most reviewers have their own personal criteria for giving perfect scores like how innovative it was or how bug free it was or whether it runs at 60 fps etc. Don't treat review scores to be an exam result or an objective measurement. It's neither of those things.
Click to expand...

I think you answer yourselfs, actual reviews aren't objective, they are subjective.

It's like music, everybody can undestand the complexity of the classical music but a lot of people can find it boring. That means actual reviews will give Justin Bieber a 10/10 and Beethoven a 6/10? Maybe, because the review will be subjective as the art it's.

Otherwise are game art? should we tread them like that? because if not we can actually measure how a good game is.
I think videogames are art, so I will be ok that some people find a game a 10/10, but for me that wouldn't be a good review, because if not I will consider to become a believer.
 
Lol, wow, the fact you unironically created this is embarrasing... It's so fucking transparent that you got worked up over the metacritic dropping.

But maybe there is some merit in having a fanboy review aggregator that only takes into account reviews that are above their sanity threshold.

you put in the game you like, put a minimum score that you'd like to see, and you get served up reviews that only match your criteria.
No more fanboys loosing their shit.

Hell, we could even put in a slideshow mode so you can pleasure yourself to the endless good reviews.
 
N.Domixis said:
What should happen, all reviews are allowed from the beginning, as soon as a a reviewer scores a 7 or below if the game is at 90+ they are banned. If a 60 or below at 80+, banned, etc. etc. Aditionally, for every 100 a game scores we are allowed to drop two of the lowest scores. If the score is 20 away from the average it's removed. All this should filter out the trolls from the authentic reviews.
Click to expand...

Quality joke post.
 
Aggregate score shouldn't really matter as different sites score differently and you have different people doing the reviews. What should matter if it matters to you is reading the content of the review and seeing if the game would be enjoyable to you or not.
 
I think getting all scores is important. Nonetheless, outliers can indeed be a problem when calculating an average score. So I think a score based in the median would solve this problem. Also, who cares.
 
Mattenth said:
I think the worst part is how bad we've done at showing you can customize your outlet selection...

The whole point of OpenCritic is that you can pick and choose which outlets you want in your score.

Metacritic uses editorial judgment over which outlets are included, and to what degree they're included (ie weighting).

OpenCritic believes that review aggregators shouldn't require editorial judgment. Period.

If the OP wants to pick-and-choose his outlets, he can do that today on OpenCritic:
Click to expand...

Forget what I've said OP. OpenCritic already does what you need.
 
hampig said:
You can't objectively review mechanics, grafix, etc. You can subjectively review your experience with these various things though.
Click to expand...

Arion said:
Everyone who writes a review elaborates the score. The written content of the review is doing exactly that.

Also please tell me how do you objectively measure graphics and gameplay.
Click to expand...

That's when I suggest to compare with other games or themes that actually people enjoy.
Because the subjective experience can't tell me a lot about the game if I don't know what do you enjoy.
 
Status
Not open for further replies.

Similar threads

  • Poll
Will Marathon Review Better Than ARC Raiders?
2
91
2K
Light_Crusader25 replied
Trails series to end in 2032, ending already decided
12
338
BennyBlanco replied
[Digital Foundry] Cyberpunk 2077 PS5 Pro Review: All New Modes & Features Tested!
Analysis Review  2 3 4
159
1K
Lagspike_exe replied
AMD Ryzen 7 9850X3D review thread- The best CPU got a little bit better
8
495
DeepEnigma replied
Invincible VS | Review Thread
Analysis Review 
0
101
Draugoth replied
Top Bottom