Support NeoGAF

[Prototype]

Lots of stuff wrong here OP

• very small sample size, no way the hand full of sites you listed are going to represent gamers from around the globe.

• which brings us to the fact that this is a very western list. Pretty much zero representation of other cultures that YOU don't approve of.

• only major outlets, so even if a site is great and produces great content and backs up what they say they have no voice?

• what basis (other than "major" site) did you use in forming this "panel" ?

• what is your background in video gaming? Writing? Reviewing? In other words, who are you to the larger gaming community? Have you written anything good? Or anything at all?


All that said and done, the timing of launching this so called project of yours is highly suspect as well. After the multiple Jim threads and polygons thread, it really makes it look like you just have an axe to grind but really didn't think it through at all. This is what I call going "full Nintendo" and it's not a good look.

 

[Nick_C]

Op your idea is bad. It's a bad idea. It's OK man, everyone shits their pants every once in a while, ya just pull up your pants, apologize for the lingering smell and the stains, and promptly leave after cleaning the couch with baby wipes and leather polish.

Spoiler

Oh and stop wining so much about dissenting review scores for games; it's sad and weird

.

poodaddy
Member
(Today, 08:06 PM)

poodaddy

 

[J_Viper]

What in the fuck man

why would you do this

 

[Arion]

Watching people actually play a game or discuss their experiences tell me more than reviews which are 1 or 2 pages long or 5 to 8 minutes in length. I like hands on videos and discussions. They give me a better idea of what the game is like and whether or not I will enjoy it. For example, watching Game Attack play Snipperclips showed me much more about what is fun and maybe not so fun about the game than listening to someone's scripted monologue about it. Strictly controlled demos less so.

Oh, you are talking about extended video footage.Fair enough but previews and hands on impressions also come as written article and short form videos.

 

[The Strokes]

I don't like the choice to only use major outlets. And gameradar is one of them?

I'm like 90% positive this exists because Jim Sterling gave Zelda a 7.

I trust him more than any of the outlets OP mentioned. Specially IGN with any Nintendo shit, the moment I see Jose Otero's reviewing I nope out of that one.

 

[SgtCobra]

Oh Jimbo what have you done

 

[Permanently A]

I think the aggregation sites are fine but articles or videos from somebody who regularly engages in trolling should not be allowed to be posted on NeoGaf. These posts and videos are toxic and do nothing but feed the troll.

When somebody comes out with an article "Why x game Is Blantantly Better Than y game" he is looking for an angered response from a certain fan base. He is looking to get DDos'ed so that he makes the news and people start sharing his "work." Suddenly, his "work" bubbles to the top of search engines and Youtube.

He is looking to become the victim so people can support him on Patreon. He says things like I don't monetize my videos or website and people drink the Koolaid that he is one of them when he is working against them. Is he really doing God's work? If he didn't get paid to be a troll, he wouldn't be trolling. If nobody fed the troll, he would just go away. You don't need to remove "critics" from Metacritic. You just need to stop talking about them.

I think the message board sites are fine but posts or videos from somebody who regularly engages in trolling should not be allowed to be posted on NeoGaf. These posts and videos are toxic and do nothing but feed the troll.

When somebody comes out with a post "He is looking to get DDos'ed so that he makes the news" he is looking for an angered response from a certain fan base. He is looking to get replies so that he makes the news and people start quoting his "post." Suddenly, his "post" is quoted and bubbles to the top of the next page.

He is looking to become the troll so people can reply to him on NeoGaf. He says things like "Is he really doing God's work?" and people drink the Koolaid that he is one of them when he is working against them. Is he really doing Kojima's work? If he didn't get paid to be a troll, he wouldn't be trolling. If nobody fed the troll, he would just go away. You don't need to remove "trolls" from NeoGaf. You just need to stop quoting them.

 

[TheChewyWaffles]

This is the review aggregator version of this poster:

Spoiler

The Guardian gave the film two stars lol

 

[Zaventem]

How about name it : MentalGymCritics ?

lol

 

[Plum]

How is that any different from a number scale? It's a 5 point ranking with 1/5 being the lowest and 5/5 being the highest. That's all a number score is. It's not a value, it's a label

Yep, aggregating a game's quality is a fool's endavour no matter the scale. Rotten Tomato's system of generating a percentage of how many critics are positive/negative about a movie, but with more nuance (i.e. instead of "fresh and rotten" have "rotten, one-to-three in-betweens, fresh") would be much preferable in my eyes.

 

[sinxtanx]

Maybe he should create an AI review aggregator that reads the reviews and aggregates the text.

this is the funniest idea I've read in a while

 

[gamerMan]

I see where you are coming from. I actually think this is a good idea. Maybe you should just cut it down to one review though- your score. So people come to your aggregate website and it asks you to input your score. You can write on the homepage that your site uses a complex algorithm to display the score. The secret formula is the score the user entered divided by 1. Let us know how we can support you on Patreon.

 

[jaypah]

I haven't read this thread all the way but is this about Zelda? Aren't people (critics included) still shitting themselves and calling it one of, if not the, greatest games ever? I don't think there will ever be a game that sits at 98 or 99 again. I just don't see how it's feasible. Just be glad that you got to play a really good game. Of if it bothers you that much just be happy that a shitload of critics voted it extremely high.

 

[Zaventem]

Fun fact Edge magazine's average review is 2 points lower than Jim's. We should remove them OP.

I haven't read this thread all the way but is this about Zelda? Aren't people (critics included) still shitting themselves and calling it one of, if not the, greatest games ever? I don't think there will ever be a game that sits at 98 or 99 again. I just don't see how it's feasible. Just be glad that you got to play a really good game. Of if it bothers you that much just be happy that a shitload of critics voted it extremely high.

RDR2 is coming.

 

[jaypah]

Fun fact Edge magazine's average review is 2 points lower than Jim's. We should remove them OP.



RDR2 is coming.

At first I thought that said R2D2 and was like, "sweet!" but yes, that's another sacred cow that will cause a ruckus too. I'm hoping the non-exclusive nature of the game will make the blow softer if it does catch some low scores. Zelda and Uncharted didn't stand a chance.

 

[TheQuickBrownFox]

Why

It's called a truncated or trimmed mean - depending on what field you're in.

https://en.wikipedia.org/wiki/Truncated_mean

 

[DarkConfidant]

One day you'll decide to READ reviews

This. Numbers are the least important part of the review, let alone the least mutually intelligible aspect when comparing multiple reviews let alone every review.

I'd be lying if I said there was value in having those random small blogs and websites counted on metacritic. I'm sure metacritic keeps them because some obscure games only get reviews from them and get ignored by major publishers, and thus would be missing a score.

You guys should also include Jimquisition, I know he's a good and decent critic.

But anyone who gives games we like numbers that aren't high enough are bad. That's right Kat of USGamer. How dare you differ from the pack!? /s

 

[ULTROS!]

You guys should also include Jimquisition, I know he's a good and decent critic.

 

[lvl99 Pixel]

Even using "trusted sites" is flawed because most outlets have multiple reviewers with very different tastes.
Might just be best to actually read reviews from critics you enjoy.

 

[inky]

This is too funny.

 

[Sanke__]

The whole point of a review aggregator is to include as many reviews as possible

You can not just reduce the amount of data to manipulate the result and pretend it is more accurate
That is very bad statistics

The only way to make a review aggregator better is find more trustworthy reviews to add to it not remove reviews

You are definitely right that removing the highest and lowest scores may have value though

 

[True Fire]

Percentages for arbitrary opinions is moronic. Gaming can't even use a Rotten Tomatoes model, because every game that isn't shit would have 98%.

The only thing that makes sense is a 5 star scale. Then BotW's controversial review would be a 3 1/2 star review, which sounds a lot less offensive than 70/100, which is a moronic way of aggregating an opinion.

 

[FingerBang]

I don't like the choice to only use major outlets. And gameradar is one of them?

I'm like 90% positive this exists because Jim Sterling gave Zelda a 7.

Literally the first thing I thought by reading the title.

 

[MyNameIsRye]

Lmao at easy allies being added. You have no criteria, other than sites you like or others like. Easy Allies hardly does reviews, and when they do they are a month late.

What a poor idea.

 

[Spladam]

Watching the Sterling fallout and reading your comments has been highly entertaining, thank you guys.

OP, I hope you realize by now that mathematically, the more variation in your aggregate, the closer to a true consensus you get. In the end, it's just opinions and has no affect or real work consequence on our, the players, actual enjoyment of the product.

If you want to really understand what you are purchasing, ignore the scores and READ the reviews.

 

[ersocaster]

How about we use all reviews, determine the standard deviation, and use that to remove outliers? Like normal stats?

 

[SHAFT MASTER]

Watching the Sterling fallout and reading your comments has been highly entertaining, thank you guys.

OP, I hope you realize by now that mathematically, the more variation in your aggregate, the closer to a true consensus you get. In the end, it's just opinions and has no affect or real work consequence on our, the players, actual enjoyment of the product.

If you want to really understand what you are purchasing, ignore the scores and READ the reviews.

Don't know what you're smoking but it's pretty clear by the outrage on this and other sites that review scores are most certainly linked to people's enjoyment of a game.

 

[danm999]

Giant Bomb isn't granular enough
Easy Allies I would maybe consider, but I don't really know about YouTubers.
No single person review sites.

Yeah heaven forbid your system which only takes seven publications into account not be super granular.

I mean you wanna talk granularity you're kind of fucked if an even number of your approved publications review a game and are all split between two scores. Guess the game has no score. Whoops.

But then this system wasn't ever meant to be used in the real world I imagine, it's a thinly veiled attempt to stan a particular game.

 

[IISANDERII]

I was under the impression that Giant Bomb was huge at this point.

They're one of the few I trust.

 

[danm999]

Any person who give a 10/10 a game simply isn't trustable as a review if it tries to be objective.

And that's because 10/10 means perfection, the best at the moment for everyone.
The reviewer has to think all the cons and pros that everybody could find, for example for Zelda you have to think about people who doesn't enjoy the graphics, or for the Dark Souls for the people who doesn't want to fight the mechanics to enjoy the game. The best of all is that these are just opinions, but opinions that some people could find..

What? No! Review scores are a metric of recommendation, not quality. When someone gets 10/10 it's not saying the game is objectively perfect with no flaws, it's a reviewer saying subjectively they can't recommend any more highly they think you should play the game.

 

[Fj0823]

What in the fuck man

why would you do this

Because Jim Sterling didn't like Zelda as hard as I did!!

 

[Permanently A]

Any person who give a 10/10 a game simply isn't trustable as a review if it tries to be objective.

And that's because 10/10 means perfection, the best at the moment for everyone.
The reviewer has to think all the cons and pros that everybody could find, for example for Zelda you have to think about people who doesn't enjoy the graphics, or for the Dark Souls for the people who doesn't want to fight the mechanics to enjoy the game. The best of all is that these are just opinions, but opinions that some people could find.

Said that in my opinion the best way to review a game is compare with others games and themes, like: if you enjoy CRPGs, liked Divinity: OS and enjoy fantasy stories, buy it. If someting of these doesn't appeal you gave it a try if you have the money and the time, if you didn't like any of these, just don't play it.

What is an objective opinion?

 

[Hazmat]

"I decided to have a cross between a fever dream and a rage stroke about some guy giving Zelda a 7/10."

 

[Outrun]

I would keep metacritic and throw out the high and low scores.

 

[danm999]

What is an objective opinion?

It'd be a system where you need as many possible review scores as video games that exist because objectivitely no two games can be of exactly equal quality.

"Did y'all see Horizon Zero Dawn got 746759037? Woah!"

 

[Rats Off To Ya]

This thread is actually hilarious. The importance people place in a fucking Metacritic score is mindblowing.

 

[Exploratory]

After rigorous scientific testing, I've compiled a list of three UNBIASED critics for my new review aggregator.

• The Jimquisition
• Gamer.no
• Slant Magazine

Horizon Zero Dawn - 95
Bloodborne - 93
The Witcher 3 - 88
The Legend of Zelda: Breath of the Wild - 63

Hahaha oh boy...

 

[catspit]

Get rid of scores. Only write ups. No scores, no stars, nothing..

 

[More_Badass]

Watching people actually play a game or discuss their experiences tell me more than reviews which are 1 or 2 pages long or 5 to 8 minutes in length. I like hands on videos and discussions. They give me a better idea of what the game is like and whether or not I will enjoy it. For example, watching Game Attack play Snipperclips showed me much more about what is fun and maybe not so fun about the game than listening to someone's scripted monologue about it. Strictly controlled demos less so.

It tells you how that section plays. That section could be the best puzzles or level in the game. It's like basing an opinion of an entire book based on part of a chapter. A review offers a retrospective look at the entire game.

A better way to pUT it is that most people's favorite games are rarely if ever the best games. So best games lists are not necessarily a good way to determine whether or not you would actually enjoy playing the game.

For example. I don't like the last of us at all. I will aknowledge it is a great game, I just don't like it. Now I see that game all over the place as being the best ever, but it's not the best ever to me, so a 10/10 means nothing to me. Where as I think kingdom Hearts 2 final mix is my favorite game and that essentially a solid 8 at best

Well, yeah, when people say something's the best anything, they mean in their opinion, out of the collection of games/books/movies/etc they've experienced

No one ever means something is the objectively best thing

 

[catspit]

I would keep metacritic and throw out the high and low scores.

Top 10% and lowest 10%

 

[Stumpokapow]

Technically ignoring statistical outliers actually can be a pretty good way of aggregating.

But eh... not like this.

But a constrained or trimmed mean (like a 0-100 score for a game) is not outlier sensitive to begin with. People seem to mostly react to occasions where a game gets 3x90 and 1x60 or whatever, but the issue there isn't the "outlier" -- it's the overall high variance owing to the small sample. Once something has 20 or 30 reviews, outliers can no longer produce leverage. (To convince yourself this is true, observe that the maximum leverage an outlier can produce at 30 reviews is 3.3% and the maximum at 20 reviews is 5%, and that's assuming the means at that point are 100% and the review it gets is 0% or vice versa, when in reality we're typically talking about something being 20-30 points too low).

Where a mean is outlier sensitive is for distributions that are unbounded, and particularly for skewed unbounded distributions. For example, if we measure the mean salary by picking 29 people from GAF and then the 30th is Mark Zuckerberg, the mean salary is $3+ million dollars -- and even if we sampled 1,000 people and 999 of them made middle class salaries and Zuckerberg was number 1000, the mean salary would be $150k+. That's when you want to start looking at trimming, outlier leverage, or using a median in lieu of a mean. Not in a distribution with ceiling and floor effects so close together.

So it seems to mostly get back to people being extremely, medically anxious about something getting an 89 instead of a 90.

 

[Skittles]

You guys should also include Jimquisition, I know he's a good and decent critic.

I find WWII to be a "good" war, in regards to what the Allied Powers accomplished under the duress applied by the Axis. It bettered humanity and changed the face of democracy and war as we know it.

You can't however, gloss over the horrible, despicable atrocities committed by the Nazis that occurred during the interim. It was a time of utter horror and crimes against humanity that none of us can, or will fully ever comprehend.

Jim Sterling's reviews are "good" - and I disagree entirely with them.

 

[rokkerkory]

What problem was this trying to solve?

 

[cheesekao]

For what purpose?

 

[IISANDERII]

https://docs.google.com/spreadsheets/d/1hO2d_BHZYT-A4JR-IQyuDoEcWGx1PevBgQsQPzh17b4/edit?usp=sharing

Explanation: This is a draft for a possible review aggregation system. If people like it I will continue to work on it.

The idea is that I only take the review scores from a few, major, well trusted gaming websites and these form a "panel" from which a score is calculated.

The panel consists of:
EDGE
Polygon
GameSpot
IGN
EGM
Game Informer
GamesRadar+

For any particular game, the highest and lowest scores are ignored and the rest are averaged to give the final score.
What do you think?

Edit: Easy Allies added preliminarily

The aggregate score your panel gave the totally busted Halo Master Chief collection at launch is 84.

4 members of your prestigious panel gave it scores of 90 or above.

 

[Rats Off To Ya]

LoL, imagine being the editor of a team of reviewers, and this guy is on your team.


9.64 points higher than average...he wants to give Horizon a 10 too.

(I currently have it at an 8.5). I just found Olin tho

It's almost like personal scales are entirely subjective and obsessing over an algorithm that tries to combine them into an objective truth is the utmost folly.

 

[Rats Off To Ya]

What problem was this trying to solve?

The existence of Jim Sterling.

 

[Permanently A]

What problem was this trying to solve?

Scientifically Trying to Eliminate Radical Leaning Influences Near Games

 

[Jumeira]

Sorry OP you don't get to decide who's good or who's trusted

 

[Flying Toaster]

What happens when Polygon drops a 7 for Zelda in the future? Are they voted off the island?