• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

There are 2 huge flaws in Steam reviews.. playtime and Steam refunds

Coreda

Coreda

Member
texhnolyze said:
You're missing my point. Here's an example to make it easier:

Whose random user's review would you trust if you have to choose between:

User A, who has 20 hours of playtime, or
User B, who has 2 minute of playtime.

Please answer.
Click to expand...

The examples in the OP aren't 2 minutes though but 40 minutes to 7.5 hours, the last one being past the refund time limit. I think Steam reviews are pretty bad in general because of the review culture that has developed there. It's not dissimilar to Reddit/Youtube's comments where users will jump on a bandwagon, post meaningless reviews for karma, make a quip, etc.
 
T

tokkun

Member
One of the problems with this proposal is that Steam's tracking of play times is not always accurate. I have several games in my library that I have beaten, but which list zero play time.

And even when accurate, the play time number may be misleading. I put 250 hours into Dark Souls on PS3, so I think I can give a reasonable review of the PC versions after only an hour.
 
S

shintoki

sparkle this bitch
Over time, I would say the reviews balance out. If it is a terrible port, then it will always be negative. If it improves. It will turn positive over time. They also have the last 30 days, to get a feeling.
 
Lanrutcon

Lanrutcon

Member
hemo memo said:
It's the same 2 hour refund limit regardless of the game. Why would anyone leave the game running for 1 hour just to shitreview? And your attuide please.
Click to expand...

Apologies.

Why does anyone shitreview? That same motivation, be it straight up malice or whatever, would be more than enough for them to just idle for an hour to get their message in the review section.

The 2 hour refund limit is there because Valve has no idea how to structure a formula that takes into account the game's actual playtime. It's already stupidly flawed, let's not repeat the mistake with another arbitrary time limit.

If we're going to pick a number out of a hat without consideration for the game's length, then fuck it. I vote for 12 minutes, 37 seconds.
 
G

gelf

Member
Only overall rating I would take any stock in is a vast overwhelming negative one. Then its probably beyond what a hate campaign over a none issue can muster. After that your just best ignoring it. A mostly negative may just be that way because some group organized against it for its "SJW themes" or maybe a bunch of ultra or bust morons have kicked it because they can't run it on maximum. And on the otherhand an overwhelmingly positive may just be because its only been reviewed by a niche group because its a low profile game, like if for example if its stacked with waifus and only thirsty anime fans have reviewed the game so far.

I'll always look elsewhere for impressions. Aggregated review scores tell me little.
 
C

CHC

Member
I was actually considering making a thread about the same thing, due in part to the harsh reviews Mankind Divided is receiving based on knee jerk reactions.

I think a 2 hour play time could fairly be enforced - i.e; the same amount of time needed for a refund. I also think it might be good to just have a short "embargo" for like 24 hours, just to let people take a breather and avoid shitting up the reviews. I think most of the really egregiously bad reviews tend show up as soon as humanly possible once the game launches.

I understand that people obviously have the right to know about technical issues, and that they also have the right to leave feedback for a product they bought, but as it is the whole system feels broken.

When reviews come down to basically being "game is shit, won't even start" from someone with 0.2 hours played, what is that really worth as actual information? Why would anyone assume that their experience will be the same?
 
B

barybll

Banned
I find the 500 hours negative reviews much more amusing

Anyway, time requirement for review is shady, since game length vary, and if you give the requirement control to devs they can easily manipulate opinions.

And even if there was such a thing as port quality separated from game review, what is stopping someone from just ignoring it and putting up a negative review because of port quality anyway, specially pissed unhappy consumers.

Valve wont do quality control of reviews, first because cutting off opinions just because you might think its wrong is a slippery slope to censorship and second because quality control isnt a process that can be automated and thus the workload would be too big
 
Bl@de

Bl@de

Member
shintoki said:
Over time, I would say the reviews balance out. If it is a terrible port, then it will always be negative. If it improves. It will turn positive over time. They also have the last 30 days, to get a feeling.
Click to expand...

I agree. Arkham Knight is at very positive now (82%) after they fixed most issues.
 
W

Wensih

Member
Sizzel said:
-Interesting point. I would argue that "quality will out" . is a 2-minute negative review worthwhile? Generally no. I am sure there is an instance, but I don't know what that is.


If a game is "good" the positive will outweigh the bad. If there is a rash of under one-hour negative reviews... something clearly is going on... either a coordinated attack on the game( which you can vet out) or the game has something that is immediately and apparently turning people off and might turn you off too so bears investigation.

the refund is there for a reason.. it incentivizes quality. Are there going to be people who play a short game to it's entirety and get a refund? yes, but the benefits of being able to return crappy games/poorly built etc outweigh those cons. short play time is a key for returns.. 100 hours isn;t likely to get a return. ...45 minutes? yes.

now.. if I was a dev whose games was getting hit by these( or an insane gamer who treated a game like my tribe) I would be emotional and hate it, but I would look closer to see what was going on.
Click to expand...

This is why I think an aggregate review score is helpful. It serves as an indicator to a community's acceptance and approval of a game at a glance. And then it would be encouraging to go find out why people are adversly reacting.
 
C

CHC

Member
shintoki said:
Over time, I would say the reviews balance out. If it is a terrible port, then it will always be negative. If it improves. It will turn positive over time. They also have the last 30 days, to get a feeling.
Click to expand...

That is true, it's still just tiring to see so many game be flooded by shit reviews on launch day. Mankind Divided is the latest and so far, based on my experiences and the concensus of the threads here on GAF, the game has hardly any readily apparent issues. Couple technical snags but absolutely nothing worth a bad review at this point.
 
T

texhnolyze

Banned
Shiggy said:
That's absolute bullshit. There are games where you can tell after a few minutes that they are shit.
Click to expand...

You haven't answered my questions..

What are the examples? Can't you see trailers or gameplay videos to come to that conclussion?

Coreda said:
The examples in the OP aren't 2 minutes though but 40 minutes to 7.5 hours, the last one being past the refund time limit. I think Steam reviews are pretty bad in general because of the review culture that has developed there. It's not dissimilar to Reddit/Youtube's comments where users will jump on a bandwagon, post meaningless reviews for karma, make a quip, etc.
Click to expand...

Right. I shouldn't have use that pic.

Whompa02 said:
I never ever trust steam reviews. Positive or not, anyone can review a game. ANYONE.

I'd rather hear an opinion from someone who isn't 12 year old Jake who loves call of duty and nothing else.
Click to expand...

It's still better than Amazon/Metacritics/etc where you can basically review anything even without buying the game.

Steam has the potential because you basically have to own the game to give a review.

CHC said:
I was actually considering making a thread about the same thing, due in part to the harsh reviews Mankind Divided is receiving based on knee jerk reactions.

I think a 2 hour play time could fairly be enforced - i.e; the same amount of time needed for a refund. I also think it might be good to just have a short "embargo" for like 24 hours, just to let people take a breather and avoid shitting up the reviews. I think most of the really egregiously bad reviews tend show up as soon as humanly possible once the game launches.

I understand that people obviously have the right to know about technical issues, and that they also have the right to leave feedback for a product they bought, but as it is the whole system feels broken.

When reviews come down to basically being "game is shit, won't even start" from someone with 0.2 hours played, what is that really worth as actual information? Why would anyone assume that their experience will be the same?
Click to expand...

Bolded: Yes, and it's been happening in most AAA games releases. People knee-jerking reviews just for fun, and even worse, troll. While it's possible for the reviews to balance out (as they normally do with most games), that kind of attitude from the community might hurt the precious first month sales.
 
R

ramparter

Banned
texhnolyze said:
Whose random user's review would you trust if you have to choose between:

User A, who has 20 hours of playtime, or
User B, who has 2 minute of playtime.

Please answer.
Click to expand...
I pity the poor soul who plays Big Rigs for 20 hours.

edit: seriously though I get your point. But imo it's still difficult to do something about it. Maybe just have a limit like 30 minutes or 1 hour. But you shouldnt force anyone to play something more than that just to give his review.
 
OmegaDL50

OmegaDL50

Member
Renekton said:
Should refund delete the review? Or put a *refunded* indicator?

Or some weighting could be put on those who played less than set time.
Click to expand...

Joyful said:
maybe solved by putting a "user has refunded" label or something I dunno
Click to expand...

This is the most sensible solution.

If a prior owner of a game left a review and refunded it. Then a "User no longer owns / refunded this game" tag next to the review in question should be appropriate.

They already tag reviews for a pre-release version of the game that may not be representative of a more recent build of the game.

I've always thought of the concept behind reviews intent to be used to educate potential buyers of the game. So a review with someone putting extensive time into the game and providing a thorough analysis would be more useful over someone buying the game, playing it for ten minutes, and leaving an impression prior to a refund doesn't seem worthwhile.

It becomes even more dubious when said review shows 0.0 hours played on the game. Almost like the the "review" in question was just the person loading it up for the first time, instantly quitting out, and making nothing more then a drive-by commentary that doesn't serve benefit but just waste valuable review space that would be better served to inform a potential buyer on the expectations from it.

Now there is also the case of games that have crashes and severe issues but these problems wouldn't just be noted in Steam reviews but on major PC gaming websites as well.

I believe a "refunded by user" tag would be reasonable.
 
W

Wensih

Member
texhnolyze said:
Bolded: Yes, and it's been happening in most AAA games releases. People knee-jerking reviews just for fun, and even worse, troll. While it's possible for the reviews to balance out (as they normally do with most games), that kind of attitude from the community might hurt the precious first month sales.
Click to expand...

Honestly, not my concern. I'd rather have the game plummet and have people not buy it in fear that it's a bad game than the reverse of dummy reviews giving positive scores and boosting the rating in the first week to buy a game. It's not an issue with refunds now, but in one scenario you avoid the game and have the ability to buy it later if you so choose in the other you were stuck with a crap game.
 
C

City 17

Member
Whompa02 said:
I never ever trust steam reviews. Positive or not, anyone can review a game. ANYONE.
Click to expand...
Only the owners of the said game, that unlike metacritic and the likes, have at least invested time and money for/in the game.

Yes, there are shallow/lulzworthy/trollish reviews among them, but it's a good system nonetheless, esp. with the addition of "recent reviews", as there are many informative/honest reviews on the general or recent state of the game from actual owners.
 
C

cyba89

Member
Accumulated ratings based on user reviews are often useless and I hate that it's the first thing you see when you look at a game page on steam. It's like slapping the metacritic user score there.

Nintendo has a time limit for user ratings on the eshop in place. I think it's two hours.
 
S

schlynch

Member
CHC said:
[...]

When reviews come down to basically being "game is shit, won't even start" from someone with 0.2 hours played, what is that really worth as actual information? Why would anyone assume that their experience will be the same?
Click to expand...

And what if 30% of the reviews are reporting this problem? Maybe it is a actual problem and denying such reviews is not a solution. On the other hand... I don't think a review after 20 hours that for example complains about the difficult of the game is more informative.

Btw, I just checked the Mankind Divided page and I think an ingame shop and microtransactions may be a pretty valid point for criticism depending on how it is implemented.
 
X

xealo

Member
Honestly, the percentage scale matters far more than any individual review, since at the end of day the system only allows for either a thumbs up or a thumbs down and nothing in between.

Games that are actually good in my experience tend to be in the 70 to 100 positive range regardless of whether some people are posting jokes in the review section.
 
M

Maintenance

Member
I had my SFV review removed after I refunded the game, never had that happen with any other game. Thank god I wasn't the only one to post against that garbage, so in the end they got fucked anyway.

So no, I do not agree with OP that you need a certain amount of hours for your review to count, since if you don't remember you can only play less than two hours before refunding. I'm not gonna get stuck with a piece of crap just to post a review.
 
Coreda

Coreda

Member
OmegaDL50 said:
It becomes even more dubious when said review shows 0.0 hours played on the game. Almost like the the "review" in question was just the person loading it up for the first time, instantly quitting out, and making nothing more then a drive-by commentary that doesn't serve benefit but just waste valuable review space that would be better served to inform a potential buyer on the expectations from it.
Click to expand...

To be fair I've personally completed some games and Steam has displayed '0' hours played. Can happen either because the player finished it in offline mode, on a different install (happened once when I played the game on OSX but posted the review on my Windows machine), or whathaveyou.
 
P

Podge293

Member
The only problem with an imposed time limit is if someone really wants to shit post a game it's not hard to leave it running while they go work or something. Hours would be clocked as "played"
 
D

Dusk Golem

A 21st Century Rockefeller
What I observe is some games suffer from 'negative bum rush' reviews, which can be an unfortunate death sentence to some indie games. Being a horror enthusiast, I see this happen to a variety of indie horror games that don't deserve it (some really good ones that some people want to post 'lol spooky memes' reviews for and try to put them in the 'garbage indie horror pile' without obviously really playing it or giving the game much of a swing), and sometimes these games do pick themselves up once a number of legitimate users play and have positive things to say, but it does leave damage as so many of these games gain their greatest exposure when they first launch, and such reviews really impact many's willingness to give it a shot. Especially for a genre in the niche.

It's the sort of risk with such a system obviously, some users/groups will abuse it, but it is what it is. Only thing one can do is raise awareness, I guess?
 
O

Ozium

Member
I think if you refund a game the review should be removed, but if you've paid for it and you keep it then you can say whatever you want about it, no matter how stupid.

I also like how they will move your friends' reviews to the top so you can have an idea of the meaning/subtext of the review since you know that person, or can always just message for elaboration.
 
B

Basileus777

Member
It can be a problem, see Dragonspear where the game got bombarded with negative reviews by a bunch of gators with no playing time.

At the very least I'd like a filter to weed out reviews from people who haven't played it enough to have an opinion of their own.
 
C

Conkerkid11

Member
Looking at that 64% for Deus Ex: Mankind Divided because not everybody can run the game at ultra settings on their integrated graphics cards while ARK: Survival Evolved has a 73% and can't run at the lowest settings with a high end rig.
 
H

Htown

STOP SHITTING ON MY MOTHER'S HEADSTONE
Ozium said:
I think if you refund a game the review should be removed, but if you've paid for it and you keep it then you can say whatever you want about it, no matter how stupid.
Click to expand...

So what you're saying is that the following user review:

Playtime: 1.5 hours
This game will not run correctly no matter what I do, there are huge technical problems.

Should be removed from Steam if they refund the game that won't work?
 
O

Ozium

Member
Htown said:
So what you're saying is that the following user review:

Playtime: 1.5 hours
This game will not run correctly no matter what I do, there are huge technical problems.

Should be removed from Steam if they refund the game that won't work?
Click to expand...

yes
 
The Scrivener

The Scrivener

Member
Imagin e if I coul d typ e.

It's a text based game. Didn't know I had to have basic reading abilities.

You thought No Man's Sky was poorly optimised? You ain't seen my mom.

This is why I hate the PC master race.
 
H

HariKari

Member
Games that get dogpiled generally deserve it for certain reasons. So these 'bad' reviews generally don't drag a game down unless there's a ton of people that have a problem with it, which is the point of the review system.
 
M

Maintenance

Member
DocONally said:
Imagin e if I coul d typ e.

It's a text based game. Didn't know I had to have basic reading abilities.

You thought No Man's Sky was poorly optimised? You ain't seen my mom.

This is why I hate the PC master race.
Click to expand...

You insulted someone's mother over a review. You ain't better than them.
 
Oppo

Oppo

Member
HariKari said:
Games that get dogpiled generally deserve it for certain reasons. So these 'bad' reviews generally don't drag a game down unless there's a ton of people that have a problem with it, which is the point of the review system.
Click to expand...

yeah but often those "reasons" are something like Platform Wars, Phil Fish Sucks, A Feminazi Made This, etc.
 
T

Tecnniqe

Banned
Ozium said:
I think if you refund a game the review should be removed, but if you've paid for it and you keep it then you can say whatever you want about it, no matter how stupid.

I also like how they will move your friends' reviews to the top so you can have an idea of the meaning/subtext of the review since you know that person, or can always just message for elaboration.
Click to expand...
Absolutely insane idea
 
I

inky

Member
They are fine. Not that much different from a forum OT where opinions are collected, except Steam aggregates them for you instead of having to guess.

How many people realisticaly buy a game just to post a review, then refund it? Isn't the fact that they refunded it giving credibility to their poor experience with it, especially if there are technical issues and whatnot?

For example, way before Steam reviews I bought a game that I enjoyed for about an hour, then it stopped working due to a bug related to my videocard that the developer knew about and refused to fix. You bet I would've both refunded it and posted a bad review even though I didn't think the game was bad for what I played.

I think people don't understand that my review is about my experience with the game, not some impossible "objective assesment of the quality" of the product or service.
 
N

Nzyme32

Member
When a game launches these seem to be overly prevalent. Meanwhile after this settles I don't see them as the norm other than during specific circumstances where people are protesting / brigading
 
L

L.O.R.D

Member
Lanrutcon said:
2 hours on record: "Game is shit"
...how can you review something you've barely played?

200 hours on record: "Game is shit"
...how can you call something bad that you played that much?

20 hours on record: "Game is shit"
...how can you call it bad when you obviously didn't play all the difficulty levels?

etc etc.
Click to expand...
I've seen something like this:
"3280 hours on record,game need optimization,it's also shit"

some people idle the game so it appears as you played that much,so no one question you or trying to be funny.
 
C

CHC

Member
schlynch said:
And what if 30% of the reviews are reporting this problem? Maybe it is a actual problem and denying such reviews is not a solution. On the other hand... I don't think a review after 20 hours that for example complains about the difficult of the game is more informative.

Btw, I just checked the Mankind Divided page and I think an ingame shop and microtransactions may be a pretty valid point for criticism depending on how it is implemented.
Click to expand...

I will agree that there is no clean solution, really - people gonna shit it up no matter what the rules are.

As for Mankind Divided, it's basically a case of people being completely allergic to the word "microtransactions." The campaign mode in the game (99% of the reason you would buy this game) remains completely unaffected - its balanced as its own game and requires no purchasing of anything (I'm not sure if you even CAN purchase things for use in the campaign). Here and there, items are hidden which you can scan with a mobile app to get some extra lore or something, but it's not like Destiny where you need them to understand what the hell is going on. It's just a silly add-in that is completely optional and often hidden. Basically, it's a non-issue but people hear "DLC" or "microtransacations" and basically go "fuck fuck fuck, fuck this, fuck you Square" etc.

In terms of DLC, yeah it has a shitty season pass, but the preorder DLC is like... nothing. Two outfits which affect nothing in-game, and a starter pack with a silenced pistol and some aug upgrades. I have it but I chose to disable it because it's that unnecessary.
 
hemo memo

hemo memo

Gold Member
Podge293 said:
The only problem with an imposed time limit is if someone really wants to shit post a game it's not hard to leave it running while they go work or something. Hours would be clocked as "played"
Click to expand...

Yes but the current one is just play for 10 minutes, ask for refund and shit on the game right away. Also they'll not leave it running becuase the 2 hours refund limit. You'll get less and less drive-by posters and that's the idea.
 
C

CHC

Member
Podge293 said:
The only problem with an imposed time limit is if someone really wants to shit post a game it's not hard to leave it running while they go work or something. Hours would be clocked as "played"
Click to expand...

Yeah it could be cheesed, but after a certain amount of (reasonable) obstacles are added, it becomes something that is really not worth doing for most people looking to shit on the game. Obviously some will still do it, but for most people flocking from reddit or 4chan to dump on a game, buying it, letting it run for 2 hours, and then trying to get a refund all in order to leave a one-line flyby review is just kind of a lot of work for little gain.
 
StereoVsn

StereoVsn

Member
schlynch said:
And what if 30% of the reviews are reporting this problem? Maybe it is a actual problem and denying such reviews is not a solution. On the other hand... I don't think a review after 20 hours that for example complains about the difficult of the game is more informative.

Btw, I just checked the Mankind Divided page and I think an ingame shop and microtransactions may be a pretty valid point for criticism depending on how it is implemented.
Click to expand...

I would agree with that. Having an in-game Microtransactions store in a SP game is shady as fuck. We are not even talking microtransaction DLCs here, it's literally shit you see in F2P games that's propagating to SP experiences.

Plus some of the technical settings are confusing for some users without much explanation and the game seems to be quite demanding while not quite having the graphical visuals for that.

cyba89 said:
Accumulated ratings based on user reviews are often useless and I hate that it's the first thing you see when you look at a game page on steam. It's like slapping the metacritic user score there.

Nintendo has a time limit for user ratings on the eshop in place. I think it's two hours.
Click to expand...
I actually think accumulated overall ratings are quite useful once things settle down (after a week or two). They are also useful when there are major problems that are immediately apparent. Plus the new "last 30 days" review option tells you about the recent state of the game.

Yeah, there are bad reviews out there but it evens out in general and you can get a feel after doing some spot checks.
 
T

texhnolyze

Banned
Htown said:
So what you're saying is that the following user review:

Playtime: 1.5 hours
This game will not run correctly no matter what I do, there are huge technical problems.

Should be removed from Steam if they refund the game that won't work?
Click to expand...

Therefore, I suggest there should be a performance or impressions instead of just reviews.
 
You must log in or register to reply here.

Similar threads

Baldur's Gate 3 Passes Half a Million Reviews on Steam and 96% Are Positive
Sales-Age
48
2K
Guilty_AI replied
Overwatch 2 director addresses review bombing on Steam
Drama Cringe Game Dev
13
1K
BlackTron replied
Paul Tassi — Positive ‘Suicide Squad: Kill The Justice League’ User Reviews Split Unusually From Critics
Opinion Analysis
44
2K
poppabk replied
DRM Enigma responds to outraged Capcom fans | Fans began to review bomb Capcom games on Steam
Drama Hardware Platform 2 3 4 5
229
17K
StereoVsn replied
The Day Before joins Overwatch 2 as one of Steam's worst-reviewed games after players discover it's not an MMO at all and extremely poorly made
Drama Community
28
2K
Gamezone replied
Top Bottom