• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Shawn Elliott's Video Game Symposium Begins, 1: Review Scores

itsinmyveins

Gets to pilot the crappy patrol labors
Brobzoid said:
WELL! that's nice. are these posted in the edge threads? I steer clear those threads because they're filled with angry people.

Yeah, there's a lot of angry folks in those threads. I don't think to many of the reviews are posted in there though. But Edge is a decent example of a magazine where you pretty much have to read the reviews in order to understand the score, and most of the time they do a good job of explaining it in the text.
 

Htown

STOP SHITTING ON MY MOTHER'S HEADSTONE
ItsInMyVeins said:
This is kinda OT, but wasn't it Kieron who wrote that awesome review of Earth Defence Force 2017 for Eurogamers a couple of years ago?

EDIT: Yeah, it was :D
That is now one of my favorite reviews ever.
 

Brobzoid

how do I slip unnoticed out of a gloryhole booth?
ItsInMyVeins said:
Yeah, there's a lot of angry folks in those threads. I don't think to many of the reviews are posted in there though. But Edge is a decent example of a magazine where you pretty much have to read the reviews in order to understand the score, and most of the time they do a good job of explaining it in the text.
the few times I'm able to read it in the store before the lady kicks me out I certainly enjoy it. Maybe I'll start a new trend, posting the texts.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Brobzoid said:
the few times I'm able to read it in the store before the lady kicks me out I certainly enjoy it. Maybe I'll start a new trend, posting the texts.

I buy it occasionally, although it comes out some time later here in Sweden than UK so most of the time I know the content and whether I'm gonna be interested in it or not. The thing about Edge is the "making of" and "time extend" sections of the magazine. Those are great. Only a few pages long, though.
 

voltron

Member
Also, I really like that idea John talked about where Rolling Stone allegedly have their editors assign scores while the writers just write it. Awesome idea. Not only would it free the writer up but it would force them to be crystal clear about what they thought of the game.
 

AndyD

aka andydumi
firehawk12 said:
I'll just summarize what I said in the comments:

When musing about having reviews on the site, the Cheap Ass Gamer proposed a dollar value scale for their reviews. So, current gen games would be reviewed out of 60 - a game that is must buy gets a 60. A game that you should rent or wait for in the bargain bit gets a 10. The idea is that the number tells you the actual dollar value of the game.

I think this is the most logical outcome for the people who believe that reviews and scores are meant to inform purchasing decisions. Rather than tell someone how much a game is worth on an arbitrary scale from 1-100 or F to A or 1 to 5 stars, tell them in cold hard numbers how much the game is worth.

At the very least the numbers become associated with a value proposition. "I think game X is worth 40 bucks" is more useful than "I think game X is a 8.4 out of 10".

Makes sense. That way we stop getting stupid reviews for $5 Live/PSN games because they aren't as full featured as a 60 dollar game...
 

Brobzoid

how do I slip unnoticed out of a gloryhole booth?
ItsInMyVeins said:
I buy it occasionally, although it comes out some time later here in Sweden than UK so most of the time I know the content and whether I'm gonna be interested in it or not. The thing about Edge is the "making of" and "time extend" sections of the magazine. Those are great. Only a few pages long, though.
isn't it like 125 SEK though? It's mad expensive outside of UK. :\ I really want to get a subscription of Retro Gamer, but it'll cost me a million.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Brobzoid said:
isn't it like 125 SEK though? It's mad expensive outside of UK. :\ I really want to get a subscription of Retro Gamer, but it'll cost me a million.

Almost. Think it's around 115:- in the store I buy it, so yeah, it's expensive. Then again it's a lot better than most of the other stuff.
 

Brobzoid

how do I slip unnoticed out of a gloryhole booth?
I do, but the lady is on to me. She starts scowling the second I walk in.
 

AstroLad

Hail to the KING baby
I subscribe to both Retro Gamer and Edge from the U.S. Generally Retro Gamer is not written nearly as well and the layout is a bit too busy, but the scope of their coverage is great.
 
that was a lot of prose to say a few things everyone with an attention span beyond that of the average gaffer already knew. yadda yadda we're not in bed with the publishers; whoa hey scores make for bad intarweb dialogue; argh i can't assign a score to something subjective. i accept this. SNOOOOOORE.

like i said -- trollishly, shokku! -- in another thread on another forum, they're ignoring the obvious pink elephant in the room: that nobody trusts videogame reviewers, given the gross inconsistency of their standards. oh, maybe we kinda sorta trust the regular byline (most likely one or more of the folks in the symposium), but we as readers inevitably turn to suspicions of bias and politics because we can't fathom why else the greater body of writers are so fucking AWFUL at reviewing, well, everything about our hobby. it's not even about scoring; it's about not being able to articulate WHY they love gta4 despite the dickslappingly obvious flaws, and why they can't meaningfully discuss a grognard wargame in a way that connects with a niche audience's expectations and demands. why should i listen to a body of critics/reviewers/commentators who can't collectively get the basics right? these accusations of bias or crookedness -- they're just the attempts by many readers to articulate a generalized mistrust.

i can open an issue of gameinformer or play or egm and find at LEAST two reviews of games where they got it fucking wrong (regardless of whether or not i liked or hated the games), and it's in three areas: an inability to dissect aaa games, the lack of desire to understand slow burn franchises, and a lack of experience with edge subgenres. the pattern match isn't pr (or the lack thereof); it isn't payola; it isn't the format requirements; it is one simple thing: reviewers, by and large, are lazy. they will not -- or cannot -- put the time or energy into thoroughly exploring every game they play, and until they address this deficiency meaningfully, they will continue to consistently earn our mistrust, rendering symposia like this largely moot outside of their own cloister. and as much as shawn and tom and n'gai and robert and stephen would like to act at some remove from the fray (and as much as they're smart enough to deserve better), they're getting tarred with the same brush, because at this point they haven't started to address the core issue of trust.

(as a sorta-aside, the one thing all of the commentators failed to note was that gamers have access to the games at roughly the same time the reviews are available. if the impressions on a forum and the review don't seem to jive, my instinct is to mistrust the reviewer, not the forum. ironically, it used to be the other way around, before the majority of my gaming diet became b-list games (having money has caused my preferences to really branch out): i'd be trying to decide if i wanted to purchase an aaa game, i'd read the unrealistically glowing impressions, and then read the reviews to curb my expectations (or have them encouraged). now, when i'm looking at the b-listers, i KNOW i can't trust that a reviewer will have spent any time with the game, or have a credible assessment of the game unto itself, much less within the sub-subgenre as it were, and instead look to forums and fellow readers to give me the straight dope.

i appreciate shawn getting together a coterie of articulate folks to chat up their problems, but man, it's just amazing that very few fingers are being pointed straight at the reviewers themselves. call some folks out! show some bad reviews! show some good ones! talk about what y'all do that fosters mistrust among your readers, and stop complaining about symptoms! (and remember that even if you're convinced your nose is one hundred percent clean -- and i'd argue in the "symposium" folks' defense that they probably have the least dirty protruberances out there -- your audience is considering the whole of game reviewing, not your individual bylines. whatcha gonna do about that?)
 

SCHUEY F1

Unconfirmed Member
AstroLad said:
I subscribe to both Retro Gamer and Edge from the U.S. Generally Retro Gamer is not written nearly as well and the layout is a bit too busy, but the scope of their coverage is great.

I was picking up EDGE on the newstand and it was horribly late all the time so I got a subscription instead and now I get it almost on time (in Canada). I think it was $150 for two years. I still like print and I enjoy the quality of the magazine.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Drinky Crow said:
now, when i'm looking at the b-listers, i KNOW i can't trust that a reviewer will have spent any time with the game, or have a credible assessment of the game unto itself, much less within the sub-subgenre as it were, and instead look to forums and fellow readers to give me the straight dope.

Then again, you're probably mainly listening to the posters you already know unless someone manages to produce a very tought through argument and seems to have played the game thoroughly.

I'm thinking that reviews would be better if they'd come out a month or so after the release though, since that'd give the review both time to play the game and let it sink in, getting some understanding of the mechanics and how they work (or not) with the game. At the same time, you can't really expect anyone to pour 50 hours into a game which is shitty from the get go.

I still prefer print magazines just because, even with their deadlines, I don't think they are under the same pressure as web based publications to get the review out asap to get all those site views.
 

AstroLad

Hail to the KING baby
SCHUEY F1 said:
I was picking up EDGE on the newstand and it was horribly late all the time so I got a subscription instead and now I get it almost on time (in Canada). I think it was $150 for two years. I still like print and I enjoy the quality of the magazine.

Same. It's a lot cheaper in the U.S. though to subscribe, 70-something I think.
 

wolfmat

Confirmed Asshole
Drinky has a point regarding the trust issue. That's why I, when I'm actually interested in a game, but need to be pushed over the edge, read lots of reviews. That, together with scanning forum banter, evens that out somewhat.

And that's totally apart from scores, mind you. I already said that in another post.

To be clear on that: There is not one particular reviewer I trust, but at times, all of them together in their similarities emphasize the core critique, the red thread. Yes, that can still totally go against my likes and dislikes, but at least I can trust that essence to be somewhat absolute in the sense that what they find appealing will appeal to a hypothetical mediocre gamer.

I don't think reviewers are necessarily lazy, but they certainly suffer from the "FIRST!!1"-syndrome, if you know what I mean, and thus often end up with early opinions as their final word on the matter, or at least that's what it seems like.

To be fair, I'm not entirely different; I can play a game for ten minutes and tell whether it's good or not according to my standards. And that's of course a premature opinion, but I wouldn't have it any other way.

But that's me and my standards, it's easy for me to understand what I want for myself, obviously.
 

Kintaro

Worships the porcelain goddess
I haven't read all of the blog since I'm here at work, but I hope the product aspect of reviews was brought up a little bit.

I think the problem with reviewing video games as opposed to movies or music is that the vast majority of people see them as "products" first. Meaning the cost still sticks out more than anything else. Perhaps that's because of their cost and because the medium is still immature. We're all guilty of it. Hell, the cost is the first thing we think of with games before anything else. It's always compared to everything the game is about.

So, perhaps this slipped into the minds of reviewers as well. Instead of being free to truly review the content and the game, they must take into consideration if the game is worth buying or not. This simple thing can skew things so much. It will also fans the flames, of course, because you have people all hyped up, ready to spend the money on a game and bam, the review score/text doesn't fit with their notion of what a $60 should be.

Maybe price should never, ever come into the heads of reviewers when playing and reviewing a game, Just get that notion out of their heads and see where their writing leads them when they don't have to worry about telling people if this game is worth their money.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Duck Amuck said:
I agree, but then when you can tell the reviewer isn't even a fan of the genre of the game and slams the game as such, there's a problem.

Yes, but you don't have to be a fan of a genre either, as far as I'm concerned. What you need is some insight and knowledge if you want to write an informed review.

About Persona 4 (or P3, actually): I'm so very tired of J-RPG:s and I can't hardly stand them nowdays. Which is why it's so weird that I'm so found of that game. Like I said, I'm not a big fan of J-RPG:s nowdays and I fucking loathe dungeon crawling but thought that P3 was one of the best games this year (released this year in EU, you know).
 

Atreides

Member
Drinky Crow said:
like i said -- trollishly, shokku! -- in another thread on another forum, they're ignoring the obvious pink elephant in the room: that nobody trusts videogame reviewers, given the gross inconsistency of their standards. oh, maybe we kinda sorta trust the regular byline (most likely one or more of the folks in the symposium), but we as readers inevitably turn to suspicions of bias and politics because we can't fathom why else the greater body of writers are so fucking AWFUL at reviewing, well, everything about our hobby. it's not even about scoring; it's about not being able to articulate WHY they love gta4 despite the dickslappingly obvious flaws, and why they can't meaningfully discuss a grognard wargame in a way that connects with a niche audience's expectations and demands. why should i listen to a body of critics/reviewers/commentators who can't collectively get the basics right? these accusations of bias or crookedness -- they're just the attempts by many readers to articulate a generalized mistrust.

Does that exist anywhere, even in reviews in other entertainment forms? Is this a defect that can be overcomed or is inherent to reviews about anything subjective?
 

itsinmyveins

Gets to pilot the crappy patrol labors
Atreides said:
Does that exist anywhere, even in reviews in other entertainment forms? Is this a defect that can be overcomed or is inherent to reviews about anything subjective?

Imho, I think that a lot of the game reviewers come off more like either disgruntled or very happy fans than critics/reviewers in comparison to music or film critics.
 
Can we drop this harping on SYMPOSIUM? The fact that word has been applied to this discussion is completely accurate!


Symposium
1. A meeting or conference for discussion of a topic, especially one in which the participants form an audience and make presentations.
2. A collection of writings on a particular topic, as in a magazine.
3. A convivial meeting for drinking, music, and intellectual discussion among the ancient Greeks.​

(Grabbed the definition from an online source because I don't have my Websters with me.)

How about we all just accept that it has been labeled a symposium and move on with consuming and discussing what results from efforts of Shawn and the other writers.
 

Atreides

Member
ItsInMyVeins said:
Imho, I think that a lot of the game reviewers come off more like either disgruntled or very happy fans than critics/reviewers in comparison to music or film critics.

That's not the point. He argued about consistency. Reviews for music or films could be better in the aspects you said. But I have seen very different reactions to the same movie from different reviewers. They are not consistent.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Atreides said:
That's not the point. He argued about consistency. Reviews for music or films could be better in the aspects you said. But I have seen very different reactions to the same movie from different reviewers. They are not consistent.

Consistency in regards of what?
 

LCfiner

Member
On another note, I find it interesting that Jeff Gerstmann, despite being in the center of the most controversial review related fiasco ever, has perhaps the most conservative and workmanlike concept of reviews on the panel. It being that the review is a simply a consumer report article.

Meanwhile, folks like N'Gai and Stephen, who spend lots of time discussing the philosophy and evolution of reviews are completely outside the insane publisher-driven pressure to overrate games or face limited access or the loss of their job.

Concerning Drinky's comments about not trusting reviewers themselves. All I can say is that podcasts and editor blogs have helped me figure out what these people like (and are like) and how to approach their written reviews.

For example, when Shane B gave Heavenly Sword a 9, I knew that he was pushing that game up based on presentation polish (also, TEH BIASSSS :). Or if Fitch reviews a JRPG for 1Up, I know he's a fan of the genre and is more likely to be familiar with its conventions. I trust him to be able to compare a subpar JRPG to a better quality one without lumping them all together as most western reviewers do.
 

Kintaro

Worships the porcelain goddess
Was there any chance to get some other game reviewers in on this one? Some folks from Gametrailers.com perhaps? I say this only because there are some folk in this talk that are partly responsible for the distrust readers have with reviews now. Some "fresh blood" may make for a more interesting discussion too.
 
Review scores are a useful heuristic, and I think most reviewers who don’t want to assign them take that position because they don’t want to deal with the public blowback their scores trigger.
 

firehawk12

Subete no aware
AndyD said:
Makes sense. That way we stop getting stupid reviews for $5 Live/PSN games because they aren't as full featured as a 60 dollar game...

If reviews are meant to inform the consumer, it should be whether or not the game is worth 5/10/15 dollars. Or as Wombat of CAG likes to put it - what would you rather have? Street Fighter 2 HD or a Big Mac?

I just noticed that some reviews of the Namco Virtual Arcade 360 release are framed that way. The disc is 30 bucks and you get Galaga and Pacman CE so they framed their review as a disc that actually cost 10 dollars because you get a 20 dollar value for the two premium games.

So why not do that for all games? "At 60 dollars, Gears 2 comes with a co-op campaign and a decent multiplayer game. But I say its only worth 50 dollars because matchmaking is broken. So if you don't need to have it, wait until you can find it for 50 bucks at Best Buy."
 

itsinmyveins

Gets to pilot the crappy patrol labors
LCfiner said:
On another note, I find it interesting that Jeff G, despite being in the center of the most controversial review related fiasco ever, has perhaps the most conservative and workmanlike concept of reviews on the panel. It being that the review is a simply a consumer report article.

What fiasco? :O

Anyhow, reviews are for the most part consumer reports (didn't Kieron Gillen mention that too, though?). That's how they're treated and that's mainly how they're written, since people mostly read the to get tips on what to buy or not.
 

wolfmat

Confirmed Asshole
Kintaro said:
Was there any chance to get some other game reviewers in on this one? Some folks from Gametrailers.com perhaps? I say this only because there are some folk in this talk that are partly responsible for the distrust readers have with reviews now.
Nope, just the Inner Circle.

I remember either Shawn or Shane jokingly referring to some popular names in the games journo biz as the Illuminati of games journalism.

It appears we now know who is at the top of this manipulative conspiracy.

23 Skidoo!
 

Atreides

Member
ItsInMyVeins said:
Consistency in regards of what?

Well, Drinky said "given the gross inconsistency of their standards". I understood it as "different people have different standards", but now that I read it again, perhaps it could mean "inconsistent standards of the same reviewer for different games".

Drinky, could you clarify what you exactly meant?
 

LCfiner

Member
ItsInMyVeins said:
What fiasco? :O

Anyhow, reviews are for the most part consumer reports (didn't Kieron Gillen mention that too, though?). That's how they're treated and that's mainly how they're written, since people mostly read the to get tips on what to buy or not.


ummm.... Kane and Lynchgate from last year? (I don't know if you're joking)

This whole discussion that Shawn is starting seems to be treating reviews as more than just consumer reports. Or, at least, understanding if there's room for reviews that are something more.

I can see the usefulness of the consumer report type. but I would also like to have a critique of a game (at least the ones experimental or interesting enough to deserve the effort) a month or two after release. Where spoilers aren't an issue and no score needs to be applied. But i think the audience for that type of article is pretty small :(

edit: Thinking about it a bit more, I already get this type of discussion from GAF and the threads discussing games like Prince of Persia or Mirror's Edge. But it would still be nice to add to this by having talented writers share their thoughts more often in longer form essays.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Atreides said:
Well, Drinky said "given the gross inconsistency of their standards". I understood it as "different people have different standards", but now that I read it again, perhaps it could mean "inconsistent standards of the same reviewer for different games".

Drinky, could you clarify what you exactly meant?

Yeah, read it like that too, and I'm guessing Duck Amuck as well since he used a comparison between those Dynasty Warrior and FIFA.

LCfiner said:
ummm.... Kane and Lynchgate from last year? (I don't know if you're joking)

This whole discussion that Shawn is starting seems to be treating reviews as more than just consumer reports. Or, at least, understanding if there's room for reviews that are something more.

I can see the usefulness of the consumer report type. but I would also like to have a critique of a game (at least the ones experimental or interesting enough to deserve the effort) a month or two after release. Where spoilers aren't an issue and no score needs to be applied. But i think the audience for that type of article is pretty small :(

edit: Thinking about it a bit more, I already get this type of discussion from GAF and the threads discussing games like Prince of Persia or Mirror's Edge. But it would still be nice to add to this by having talented writers share their thoughts more often in longer form essays.

Oh, shit, I read "Jeff G" as Jeff Green. My bad.

The thing with consumer reports is that they're probably more likely to catch interest than an in depth analysis of a game, which of course impacts the sales of ads and so on.

The things you're asking for is what I think I get from reading, like earlier mentioned, the "Making of" (very old games) and "Time Extend" (last generation old games) parts of Edge magazine.
 

LCfiner

Member
ItsInMyVeins said:
Yeah, read it like that too, and I'm guessing Duck Amuck as well since he used a comparison between those Dynasty Warrior and FIFA.



Oh, shit, I read "Jeff G" as Jeff Green. My bad.

The thing with consumer reports is that they're probably more likely to catch interest than an in depth analysis of a game, which of course impacts the sales of ads and so on.

The things you're asking for is what I think I get from reading, like earlier mentioned, the "Making of" (very old games) and "Time Extend" (last generation old games) parts of Edge magazine.


:lol :lol oh shit, that's good.

I edited my original post. I hadn't made that connection since Jeff Green wasn't part of the symposium (but he is on our minds, I guess, with the recent GWJ cast)

EDGE articles are great but it's too pricey whenever I (rarely) see it. 15 bucks CDN. ugh. I do try to read the articles online when they show up, though.
 
Atreides said:
That's not the point. He argued about consistency. Reviews for music or films could be better in the aspects you said. But I have seen very different reactions to the same movie from different reviewers. They are not consistent.

that's not what i meant by consistency, in this case. rather, i can by and large assume that all reviewers of movies, books, music, and art have a breadth of knowledge that allows them understand -- not like, understand -- what the various approaches, methods, styles, genres, and movements are within the greater medium. i don't get that from videogame reviewers.

would you trust music reviews if half of them bashed hip-hop as random, disinteresting noise? if sci-fi magazines regularly mocked your favorite pulp fantasy writer for writing about elves and orcs, instead of evaluating how well the writer approached the trope? if art critics across the board didn't know who degas was, or had no fundamental understanding of impressionism, yet coughed up hyperbolic mainstream reviews nonetheless?

i can trust that most movie reviewers can, for example, watch a haneke film and while they may hate it, they will be able to talk about it credibly and meaningfully. likewise, a literary reviewer can read a sci-fi novel and dissect it based on the craft, rather than dwelling on trope or genre. i can NOT expect that from the current crop of game reviewers. the symposium folks are better than most, but if they're gonna effect change instead of hemming and hawing and simply agreeing with each other (by and large), they need to call their own PRACTICES to task in front of their whole business. like i predicted in the qt3 thread, there's no sense of OWNERSHIP for these issues coming from them. (maybe that's just my manager experience talking, though, and perhaps it's foolish for me to expect a bunch of well-known game bloggers casually changing opinions to seek results, despite the semi-formal presentation of it all.)

i don't much like modern chick lit, for example, but if you ask me to review janet evanovich's latest, you better damn well believe i'll dive into her backlog, despise it as a may, just to make sure i have a credible and well-formed opinion. thankfully, i would hope to never be in such a position. likewise, i wonder why the folks who hate dungeon crawlers, football management sims, and dynasty warriors games are in THEIR positions!
 

itsinmyveins

Gets to pilot the crappy patrol labors
Drinky Crow said:
the symposium folks are better than most, but if they're gonna effect change instead of hemming and hawing and simply agreeing with each other (by and large), they need to call their own PRACTICES to task in front of their whole business.

But isn't that what they're trying to do though? They are talking about the approach the reviews as a whole -- writing consumer reports and etc -- not just how ads and covers come into play when getting exclusives and scoring etc?
 
ItsInMyVeins said:
But isn't that what they're trying to do though? They are talking about the approach the reviews as a whole -- writing consumer reports and etc -- not just how ads and covers come into play when getting exclusives and scoring etc?


well, maybe that's just my manager experience talking, and perhaps it's foolish for me to expect a bunch of well-known game bloggers casually exchanging opinions to seek results despite the semi-formal presentation of it all, but as i edited in above, i don't see any real ownership of these problems, which bugs me.

hey, there's a lot of good thoughts in the whole thing, and kieron's remark about how aaa games start at 9/10 and get marked down whereas the b-listers start at 5/10 and hafta work up was right on the money and great to have written down, but that wasn't the way the conversation turned -- it turned to complaints about "superfans" and their expectations; about the non-influence of pr firms and the implied heroic stands taken against it; and the pointlessness of ascribing numbers to experience. (shawn tried to turn it back to no avail.)

tom at least admitted that the idea of context is completely valid; that reviewers don't review in a vaccuum, and that it's very hard -- or perhaps even undesirable -- to move outside that context. THAT bears examining. if we can't trust a similar context for most reviewers -- or at least get a little disclosure from time to time -- well then, readers and reviewers are gonna hafta part ways. i bash dave halverson a lot, but at least he admits his insane preferences and his inability to develop a breadth of appreciation.

i just wanna see someone acknowledge that there's a trust issue here. given that the symposium set consists of several folks who have probably earned actual trust from their readers, they're the safest ones to observe it -- am i gonna think less of tom or stephen if they say "hey, it's hard to get a review done on time and fully evaluate it, especially when i have 10 other games in the queue and this one's a game from a franchise i don't get in a genre i don't care about"? probably not. so why do they talk around it -- because it might suggest that their own bodies of work aren't so unimpeachable? because they feel it might cast a bad light across the discipline of game reviewing as a whole? or because i'm wrong and every reviewer DOES spend time playing through a significant chunk of games like nightmare of druaga or grand theft auto 4? just askin' is all.
 

FartOfWar

Banned
Drinky Crow said:
it's not even about scoring; it's about not being able to articulate WHY they love gta4 despite the dickslappingly obvious flaws, and why they can't meaningfully discuss a grognard wargame in a way that connects with a niche audience's expectations and demands. why should i listen to a body of critics/reviewers/commentators who can't collectively get the basics right? these accusations of bias or crookedness -- they're just the attempts by many readers to articulate a generalized mistrust.

I'll respond as I read. You're arguing that I'm unable to articulate what I like and dislike about games and why?
 

sikkinixx

Member
Interesting read about two-thirds through. It's odd to read though as it feels academic in tone (well except the occasional "bullshit" added in for flavour) so the academic in me wants examples, citations, sources. I want to SEE what reviews are using the clichés that everyone tip-toes around. I know it's not kosher to do that but still.


This bit by Robert Ashley:

"I took a break from enthusiast press game reviews for a couple of years. What a fucking relief. No more death threats from insane superfans who think my evaluation of their favorite game is some kind of paid-for hit job by a shadowy corporate network."

makes me hope that perhaps in the "Reader Backlash" segment of the "symposium" will address how reviewers feel in their job. What influence the super fanboys might have on the writing. Shawn and Jeff Green talked in the GWJ podcast about the role ego plays in running a (successful) podcast, and the positive feedback ("I'M JEFF GREEN!!") and it makes me wonder if reader pressure effects how games are reviewed. Maybe some reviewers want to cause some commotion like some readers accused Jeff Gerstmann did with his 8.8 to Twilight Princess? (I am not arguing Gerstmann did, only that many Zelda fanboys thought he was) Maybe some just give into the hype surrounding games, as Hilary Goldstein did in his GTAIV review with: "The story is Oscar quality. The use of the phone as a gaming portal is genius. There's really nothing more that could be asked for from GTA IV" ?

Moreover, one thing I would love to hear discussed at some point (if it can fit into a discussion) is regarding qualification to do the job gaming journalists have. Leigh Alexander points out that "the anxiety I get sometimes knowing people's jobs depend on, say, Metacritic, and hoping that I was as thorough and fair as I could possibly be" when thinking about scores can have some stress. Reading that made me wonder what gives gaming media the qualifications to do this? Gaming is somewhat unique in that you have to be somewhat good at playing games to properly critique them whereas with say music, you don't need to be able to play guitar in order to critique the latest _______ album. I know many gaming media folk have went to college and such now or have writing experience, which I suppose is a credential for doing reviews and the like. I suppose I feel bad for developers at time because I envision them saying "well who the fuck is this 22 year old with a bachelors degree in computer science to ruin my bonus by giving me a shit score?"


Good job though Shawn, thanks for doing this as even just this first bit has been an insightful read.
 

Atreides

Member
Well Drinky, is not like they have finished this. There are eight parts. This is only the first one, and it's about review scores. I would wait until they have finished before saying that they haven't talked about that problem.
 

LCfiner

Member
Atreides said:
Well Drinky, is not like they have finished this. There are eight parts. This is only the first one, and it's about review scores. I would wait until they have finished before saying that they haven't talked about that problem.

yup, although I do think it's good to bring it up as it might influence the future portions and perhaps they could speak more about reviewer "trust" as Drinky mentions.
 
FartOfWar said:
I'll respond as I read. You're arguing that I'm unable to articulate what I like and dislike about games and why?

man, i've been careful to qualify that not EVERY reviewer has these problems. you're looking at your body of work; i'm looking at the body of work out there across 40-50 reviewers at the major magazines and sites.

look. i can pick up game informer, or go to ign, and immediately find a review that gets a game completely and totally wrong. does that mean that its YOU that's spinning your share of the dross out there? of course not, but when you're a reader, the byline is secondary.
 

FartOfWar

Banned
Drinky Crow said:
that was a lot of prose to say a few things everyone with an attention span beyond that of the average gaffer already knew. yadda yadda we're not in bed with the publishers; whoa hey scores make for bad intarweb dialogue; argh i can't assign a score to something subjective. i accept this. SNOOOOOORE.

like i said -- trollishly, shokku! -- in another thread on another forum, they're ignoring the obvious pink elephant in the room: that nobody trusts videogame reviewers, given the gross inconsistency of their standards. oh, maybe we kinda sorta trust the regular byline (most likely one or more of the folks in the symposium), but we as readers inevitably turn to suspicions of bias and politics because we can't fathom why else the greater body of writers are so fucking AWFUL at reviewing, well, everything about our hobby. it's not even about scoring; it's about not being able to articulate WHY they love gta4 despite the dickslappingly obvious flaws, and why they can't meaningfully discuss a grognard wargame in a way that connects with a niche audience's expectations and demands. why should i listen to a body of critics/reviewers/commentators who can't collectively get the basics right? these accusations of bias or crookedness -- they're just the attempts by many readers to articulate a generalized mistrust.

i can open an issue of gameinformer or play or egm and find at LEAST two reviews of games where they got it fucking wrong (regardless of whether or not i liked or hated the games), and it's in three areas: an inability to dissect aaa games, the lack of desire to understand slow burn franchises, and a lack of experience with edge subgenres. the pattern match isn't pr (or the lack thereof); it isn't payola; it isn't the format requirements; it is one simple thing: reviewers, by and large, are lazy. they will not -- or cannot -- put the time or energy into thoroughly exploring every game they play, and until they address this deficiency meaningfully, they will continue to consistently earn our mistrust, rendering symposia like this largely moot outside of their own cloister. and as much as shawn and tom and n'gai and robert and stephen would like to act at some remove from the fray (and as much as they're smart enough to deserve better), they're getting tarred with the same brush, because at this point they haven't started to address the core issue of trust.

(as a sorta-aside, the one thing all of the commentators failed to note was that gamers have access to the games at roughly the same time the reviews are available. if the impressions on a forum and the review don't seem to jive, my instinct is to mistrust the reviewer, not the forum. ironically, it used to be the other way around, before the majority of my gaming diet became b-list games (having money has caused my preferences to really branch out): i'd be trying to decide if i wanted to purchase an aaa game, i'd read the unrealistically glowing impressions, and then read the reviews to curb my expectations (or have them encouraged). now, when i'm looking at the b-listers, i KNOW i can't trust that a reviewer will have spent any time with the game, or have a credible assessment of the game unto itself, much less within the sub-subgenre as it were, and instead look to forums and fellow readers to give me the straight dope.

i appreciate shawn getting together a coterie of articulate folks to chat up their problems, but man, it's just amazing that very few fingers are being pointed straight at the reviewers themselves. call some folks out! show some bad reviews! show some good ones! talk about what y'all do that fosters mistrust among your readers, and stop complaining about symptoms! (and remember that even if you're convinced your nose is one hundred percent clean -- and i'd argue in the "symposium" folks' defense that they probably have the least dirty protruberances out there -- your audience is considering the whole of game reviewing, not your individual bylines. whatcha gonna do about that?)

Drinky, we will discuss several of your concerns in forthcoming sections. I guarantee it. As for calling out the crap -- I did do that on GFW Radio. Now that I'm making a game that some of the same people will review, I probably shouldn't start that shit. Not unless I want my boss to ream me. Maybe others might, provided there's a point to it. Then again, I think that the inadequate analysis characteristic of game reviews in general should be obvious.
 
LCfiner said:
yup, although I do think it's good to bring it up as it might influence the future portions and perhaps they could speak more about reviewer "trust" as Drinky mentions.

hey, i'm just talkin'. i fully allow that they might blow my fuckin' mind and i'll be laying here all fetal and moaning "halo deuce" to the lot of you, but this a videogaming forum, not a symposium. i talk off the cuff; i don't prepare my remarks. fuck, i don't even spellcheck.
 

itsinmyveins

Gets to pilot the crappy patrol labors
Drinky Crow said:
hey, there's a lot of good thoughts in the whole thing, and kieron's remark about how aaa games start at 9/10 and get marked down whereas the b-listers start at 5/10 and hafta work up was right on the money and great to have written down, but that wasn't the way the conversation turned -- it turned to complaints about "superfans" and their expectations; about the non-influence of pr firms and the implied heroic stands taken against it; and the pointlessness of ascribing numbers to experience. (shawn tried to turn it back to no avail.)

Well, I'm just gonna go ahead again and say that, imho, one of the problems with most of the game reviews stem from the reviewers being "superfans" or just lacking distance to the game(s). It's the opposite of what you mentioned about movie and music reviewers who often can form a coherent text about something they don't like very much. But if this is solely because of reviewers or the fact that games do differ from both movies and books and music, or to what extent, I don't know man. Maybe what readers expect and want is also a factor?

Drinky Crow said:
i just wanna see someone acknowledge that there's a trust issue here. given that the symposium set consists of several folks who have probably earned actual trust from their readers, they're the safest ones to observe it -- am i gonna think less of tom or stephen if they say "hey, it's hard to get a review done on time and fully evaluate it, especially when i have 10 other games in the queue and this one's a game from a franchise i don't get in a genre i don't care about"? probably not. so why do they deny it -- because it might suggest that their own bodies of work aren't so unimpeachable? because they feel it might cast a bad light across the discipline of game reviewing as a whole? or because i'm wrong and every reviewer DOES spend time playing through a significant chunk of games like nightmare of druaga or grand theft auto 4? just askin' is all.

Of course there's a trust issue. There's deadlines, reviewers who go to companies to play the game for a bunch of hours and review based of that, a lack of knowledge and so on. Sometimes reviewers seem to be in a contest to come up with the snappiest closing comments on that latest "must be an AAA-game!". But I'm not gonna expect a reviewer to play through a 40h game which sucks from the start, but they should have some understanding of the core concept and basics of that game.
 

FartOfWar

Banned
Drinky Crow said:
man, i've been careful to qualify that not EVERY reviewer has these problems. you're looking at your body of work; i'm looking at the body of work out there across 40-50 reviewers at the major magazines and sites.

look. i can pick up game informer, or go to ign, and immediately find a review that gets a game completely and totally wrong. does that mean that its YOU that's spinning your share of the dross out there? of course not, but when you're a reader, the byline is secondary.

I misunderstood. I somehow got the sense that you see one writer as all of his brothers' keepers.
 

Flavius

Member
Drinky Crow said:
of course not, but when you're a reader, the byline is secondary.

Shows how out of touch I am, I suppose. For me, the byline is nearly everything. I seek out those reviewers/enthusiasts/whatever-the-hell-you-want-to-call-them whose body of work I respect. If the name's unrecognizable, I don't immediately dismiss it as shit, but I will seek out that person's other work to get a sense of their approach...skills, ethics, and whatnot.
 
Top Bottom