-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
Metacritic's weighting system revealed
- Thread starter RotBot
- Start date
Planet Xbox 360 and Xboxic are in there too.
Cosmo Clock 21
Banned
In case people might have thought otherwise... sites didn't know their own weight, much less publishers.
Edit: is the precise methodology applied to find the weights published anywhere? Wanna see those cases where the weighting failed and apply hypothesis testing to them.
Edit: is the precise methodology applied to find the weights published anywhere? Wanna see those cases where the weighting failed and apply hypothesis testing to them.
AlphaTwo00
Member
And your resume!
Man. I loved answering to employers and recruiters about my last 3 game's metacritic scores!
I like to have faith that they didn't. Besides that, none.
Now that'd be interesting to see...
Technosteve
Banned
i find funny firing squad is 1.5 weighted
Green Yoshi
Member
I was so wrong. ^^
nckillthegrimace
Member
No, it is based on quality. If this list is correct, it represents Metacritic's beliefs as to what the best review sites are.
Metacritic said:
Which explains this: http://www.metacritic.com/faq#item23
Metacritic said:
Having the list out in the open exposes the whole thing as a nonsensical farce.
MoogleWizard
Member
Nintendo don't pay, they don't give a shit. Not one of their mags is 1.5 while several Playstation mags are. Lol, what a biased joke of site.
XANDER CAGE
Member
I actually had no idea that there was such a huge disparity in the way they weighted things. I'd love to see similar data on how they handle film reviews, too. At least RottenTomatoes is fairly upfront about categorizing "cream of the crop" publications.
I propose the creation of a new rankings website where, instead of weightings, the distribution of scores given by websites is normalised based on their mean review score and then spread in a normal distribution about 50%.
You could also go finer grain and do it per individual reviewer but that might lead to bias. For instance, some sites like to have reviewers who focuses on AAA titles which typically get higher scores and freelancers who review low-scoring shovel ware.
Obviously, the need for such a rankings system is questionable, but if there must be one, it can be done more fairly than Metacritic.
You could also go finer grain and do it per individual reviewer but that might lead to bias. For instance, some sites like to have reviewers who focuses on AAA titles which typically get higher scores and freelancers who review low-scoring shovel ware.
Obviously, the need for such a rankings system is questionable, but if there must be one, it can be done more fairly than Metacritic.
It really has lost all credibility and usefulness. Hoping the thing becomes abolished during this next gen.
timetokill
Banned
Systemic anti-Nintendo bias confirmed
synchronicity
Member
Yeah, it seems really unnecessary. Either use a particular critic's/site's reviews or ignore them altogether. That said, I still think metacritic is a pretty good general reference for overall opinion. Most of the time user/ critic average scores don't differ too wildly. (I know there are exceptions.)
Well, it kinda makes sense as you can have real sleazy sites that give shitty ass reviews for absolutely everything and you want some way to protect against that. At the same time it's bullshit because good quality publications get ignored (e.g. Giant Bomb) in favour of mainstream sites and we don't exactly know why this is. Is it because of readership size? Or did they pay CNET? Or is it their based on their history?
Who knows, and to be honest I don't give a shit
Who knows, and to be honest I don't give a shit
I second the motion.
Sony paid (the most?).
Aquamarine
Member
I still can't believe that Metacritic tallies IGN THREE TIMES.
Effectively, the IGN umbrella has a 4.5 weight....8 times higher than Giant Bomb.
Now this just makes me angry.
Effectively, the IGN umbrella has a 4.5 weight....8 times higher than Giant Bomb.
Now this just makes me angry.
UberTag
Member
Not quite... otherwise you'd see some games scoring above 100%.
What it means is that their 9.4 carries the weight of 1.5 reviews in the average.
Whereas the Giant Bomb 5 out of 5 only carries the weight of 0.5 reviews in the average.
SolarPowered
Member
Wow brb banning metacritic from my browser.
Domstercool
Member
Wow O_O I write for DarkZero, which is a volunteer based site (we all have other jobs or are students, etc.)
I have no idea how we are fixed into the highest category. O_O I'm shocked. We don't get paid or anything, we're just a group of guys who like gaming. O_O
I have no idea how we are fixed into the highest category. O_O I'm shocked. We don't get paid or anything, we're just a group of guys who like gaming. O_O
Perhaps you should ask Obsidian about that.
Wiz
Member
Mama Robotnik
Member
Gaming industry discussion on a gaming forum, what gives?
Seriously, this data is a fascinating insight into just how truly nonsensical and undeserved the power of MetaCritic is.
Ah I see.
CosmicQueso
Member
Well, guess who isn't getting a Christmas card from Metacritic this year? I'm also guessing his next research request might not be approved. Just a hunch!
BTW... Full Sail University? What's this school's cred like? I heard they beat SDSU in the tourney. Oh... wait a minute...
Aquamarine
Member
It's just annoying because LEGITIMATE PUBLISHERS use this bullshit as a metric for company bonuses. :-(
timetokill
Banned
You just described 99% of game journalists/critics
People who have their jobs and bonuses rely off said scores, for starters
That's what they say. But looking at the list, I don't think many people would agree with what they determine as quality and stature. So either:
1) They have an incredibly skewed sense of quality; some sites are unusually high, some are unusually low.
2) They're lying and they weigh publications by a different criteria.
I don't want to say for sure it's 2, because I have no evidence to suggest they're lying (and because of that, it's easier to believe they just have a poor barometer of quality), but something's definitely up. Whether that's a farce or something more sinister, I don't know!