• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Metacritic's weighting system revealed

Sinatar

Official GAF Bottom Feeder
Not quite... otherwise you'd see some games scoring above 100%.

What it means is that their 9.4 carries the weight of 1.5 reviews in the average.
Whereas the Giant Bomb 5 out of 5 only carries the weight of 0.5 reviews in the average.

Yea but as mentioned above you IGN get's tallied 3 times. So their 9.4 is actually a 28.2.

This thread should honestly be exploding, this is insane.
 
What a broken system. I hope more people start ignoring metacritic. Such a terrible site.

Someone needs to email every major publisher and tell them all the stupidity behind these weight rankings.

Metacritic SHOULD NOT, IN ANY RESPECT be used as a tool to determine company bonuses.
 
It's just annoying because LEGITIMATE PUBLISHERS use this bullshit as a metric for company bonuses. :-(

Its just a deeper layer to the joke. MetaCritic is nonsense, using nonsense weightings to produce nonsense data, which is held as sacrosanct by billion dollar video gaming megacorps.

If the data wasn't being used to withhold bonuses, and close studios, it would be funny.
 

Fezzan

Unconfirmed Member
Really?

Really?
Um.... yes.


Gaming industry discussion on a gaming forum, what gives?

Seriously, this data is a fascinating insight into just how truly nonsensical and undeserved the power of MetaCritic is.

Metacritic is sorta pointless, I use it but I don't look at the numbers.
I don't use it for all games but if I don't know about one and it's green I will get it, orange more research needed and red just pass unless I think I will enjoy it.
It's useful but meh.
 

Dahbomb

Member
Who cares?
A lot of publisher and company actions and decisions are based on MC. Game developer which makes a game that gets high MC will get bonuses and future games greenlighted (if they sold decent).

Gamers don't nearly care as much about MC as publishers do.
 
Yeah, seeing how a volunteer site that I've contributed to is weighted on here is... bizarre. I don't like it.

Also seeing Giant Bomb at a .5 is a goddamn tragedy.
 

sixghost

Member
Someone needs to email every major publisher and tell them all the stupidity behind these weight rankings.

Metacritic SHOULD NOT, IN ANY RESPECT be used as a tool to determine company bonuses.

Publishers probably love this. Now they know who to focus their attention on.
 

Yagharek

Member
Someone needs to email every major publisher and tell them all the stupidity behind these weight rankings.

Metacritic SHOULD NOT, IN ANY RESPECT be used as a tool to determine company bonuses.

Agree 100%, but let's not fool ourselves. Publishers love an easy metric to judge a game's success. Everything is developed according to formulae these days.
 

ZealousD

Makes world leading predictions like "The sun will rise tomorrow"
Is there a chance that the weighting is based off something like the variance between the site's individual scores compared to the unweighted metascores or something like that? That way publications which adhere to the consensus get higher weights.? That's the only explanation I can think of.
 

Brashnir

Member
To be fair, gamerankings aggregates are barely any different than metacritic's, so the weighting doesn't end up having much of an effect. I'm not sure why they even bother.

aren't gamerankings and metacritic both under the CNET umbrella? Who's to say they even use different formulas.
 

UberTag

Member
Yea but as mentioned above you IGN get's tallied 3 times. So their 9.4 is actually a 28.2.

This thread should honestly be exploding, this is insane.
I'm not sure there are many games where multiple IGN scores are counted. That's certainly not the case for Tomb Raider or Bioshock Infinite.

Both IGN UK and IGN reviewed Xenoblade Chronicles and have unique scores due to it having a staggered release. So they're effectively carrying a weighted 3.0 influence on that game's Metacritic score.
 

DocSeuss

Member
Highest (1.5) -- GamePro
Lower (0.5) -- Giant Bomb

oh

I think Metacritic takes GiantBomb in increments of 20, since they have a five point system.

What they should do is equate GiantBomb's scores with an ABCDF 95/85/75/65/55% system, so a 4/5 would be an 85, and a 3/5 would be a 75. Instead, I think they consider a 3/5 to be a 60, and a 4/5 to be an 80.

Basically, Metacritic is stupid.

But also, Metacritic tends to weight critics based on how in line they are with everyone else, which is a stupid way to rate games.

I wonder what New Vegas's score would be if their grades were "unskewed."

I'll try to run the math later, if I'm not busy, but I think they'd actually be lower.
 

MrT-Tar

Member
I propose (without a doubt by no means the first to say this) that Metacritic should be a banned site.

This has simply confirmed what I expected/dreaded
 

Orayn

Member
Um.... yes.




Metacritic is sorta pointless, I use it but I don't look at the numbers.
I don't use it for all games but if I don't know about one and it's green I will get it, orange more research needed and red just pass unless I think I will enjoy it.
It's useful but meh.

Most people in this thread agree with you. Metacritic is given undue weight, and worse still, it gets its numbers through some pretty bullshit methods. This would only be mildly annoying, were it not for the fact that publishers throw around fat stacks of cash to ensure a certain Metacritic score, or only pay developers royalties on their game if it gets above a certain score on MC. It's madness.
 

Eusis

Member
People who have their jobs and bonuses rely off said scores, for starters
This. "All fun and games until someone loses an eye", and we've had cases like Obsidian missing out on a bonus because of a Metacritic average. Ironically enough, if GameRankings treats all scores equally that means Metacritic gave them a better chance, though depending on 85%+ when many don't have a matching scores (only 10 or 5 points) is kinda crap anyway.
 

majik13

Member
so there are no Nintendo websites in the High and Highest category, but plenty of PS and Xbox only websites in those categories.

mmmm...ok.

why even do weighting?
 

Fezzan

Unconfirmed Member
Yea but as mentioned above you IGN get's tallied 3 times. So their 9.4 is actually a 28.2.

This thread should honestly be exploding, this is insane.

Actually there all unique scores.
IGN might give Bioshock infinite a 9.5 but UK might give it a 9.
At least that's how it worked when I last went there.
 

Miles X

Member
Um.... yes.




Metacritic is sorta pointless, I use it but I don't look at the numbers.
I don't use it for all games but if I don't know about one and it's green I will get it, orange more research needed and red just pass unless I think I will enjoy it.
It's useful but meh.

You're pretty dumb if you have to ask that question A) because it's basically thread whining and B) all the reasons listed above. Meta IS dumb but it's sometimes the decider between if a dev gets a $$$$ bonus or not.
 

kubus

Member
Wow O_O I write for DarkZero, which is a volunteer based site (we all have other jobs or are students, etc.)

I have no idea how we are fixed into the highest category. O_O I'm shocked. We don't get paid or anything, we're just a group of guys who like gaming. O_O
Meanwhile, InsideGamer is ranked lowest while it's the biggest game website in the Netherlands. Much smaller Dutch websites are in the highest rank.

Metacritic:

wpid-i_have_no_idea_what_i_m_doing.jpg
 

Randdalf

Member
For those wondering how it might work.

What you do, is you total the contribution of each site. For example:
Code:
GamesLol + GamesLmao + IGN = 1 + 1 + 1.5 = 3.5.

You then normalise the weighting based on the total, so:
Code:
GamesLol = 1/3.5 = 0.28
GamesLmao = 1/3.5 = 0.28
IGN = 1.5/3.5 = 0.43

Then when calculating the scores:
Code:
metascore = score1*GamesLol + score2*GamesLmao + score3*IGN
metascore = 75*0.28 + 75*0.28 + 93*0.43 = 82
With just a plain average:
Code:
(75+75+93)/3 = 81

I assume it's more complex of a model than that, but that's the idea of weightings. It just lends more weight to that site's opinion, so a site with a higher weighting is basically more influential on the score. In my example, IGN is more influential than the other two sites, but in reality it has only a small impact on the actual metascore. When calculated with many different sites, as Metacritic does, I don't expect it actually makes an enormous amount of difference.
 

Fezzan

Unconfirmed Member
Most people in this thread agree with you. Metacritic is given undue weight, and worse still, it gets its numbers through some pretty bullshit methods. This would only be mildly annoying, were it not for the fact that publishers throw around fat stacks of cash to ensure a certain Metacritic score, or only pay developers royalties on their game if it gets above a certain score on MC. It's madness.

The fact that publishers use it for firing people and enticing people to buy games is stupid, I agree.
 
Most people in this thread agree with you. Metacritic is given undue weight, and worse still, it gets its numbers through some pretty bullshit methods. This would only be mildly annoying, were it not for the fact that publishers throw around fat stacks of cash to ensure a certain Metacritic score, or only pay developers royalties on their game if it gets above a certain score on MC. It's madness.

This. If they want to weight this way that's their prerogative, but they have to be aware of how important they've become to the development side of things. Honestly, I'd say they have a responsibility to change their system now that they're the go-to site for score aggregation.

And that's the thing, I don't think aggregation is a bad thing. It's just that it sort of loses a lot of meaning when you skew numbers so (seemingly) randomly, not to mention crazy stuff like the way it's affected bonuses, budgets, etc.
 

Fezzan

Unconfirmed Member
You're pretty dumb if you have to ask that question A) because it's basically thread whining and B) all the reasons listed above. Meta IS dumb but it's sometimes the decider between if a dev gets a $$$$ bonus or not.

It wasn't exactly a serious question...
 

Yonafunu

Member
Uuuhhm, why are there several (official) Playstaion and Xbox publications in the highest two categories, but no Nintendo publications whatsoever?
This is obviously not the biggest question here, but it stood out to me.
 

Noof

Member
Wow. I don't understand why they would do that. Publishers are going to be throwing money at the sites in the highest tier.
 
That's what they say. But looking at the list, I don't think many people would agree with what they determine as quality and stature. So either:

1) They have an incredibly skewed sense of quality; some sites are unusually high, some are unusually low.

2) They're lying and they weigh publications by a different criteria.

I don't want to say for sure it's 2, because I have no evidence to suggest they're lying (and because of that, it's easier to believe they just have a poor barometer of quality), but something's definitely up. Whether that's a farce or something more sinister, I don't know!

I'm thinking it might be weighted by accessible data... Site readership, analytics data, or maybe click throughs from their site to external review sites.

I think this does make a good case for a rival service.
 
Not sure why people are surprised by giantbombs weight, seems obvious they would be in the bottom with a 5 star system that often is much lower than the rest when translated to meta critic BS.
 

newjeruse

Member
Haha, wow, Metacritic is a joke.

Nah, it's not. It's a pretty useful site actually. If you want to blame someone, blame the developers for tying bonuses to such of a back-of-the-envelope calculation. Metacritic is cool. They're not out to get anyone.
 
Top Bottom