texhnolyze
Banned
Retroactive. You can already see the result now.
-
Hey Guest. Check out your NeoGAF Wrapped 2025 results here!
Steam Customer Review system updated (default score only includes Steam buys)
- Thread starter MUnited83
- Start date
Retroactive. You can already see the result now.
Metal Gear?!
Member
Guess the game!
PepsimanVsJoe
Member
d00d3n
Member
Raising the mean score of their game, I guess. That gets more difficult when you have negative votes in the 10000s, though.
Why? I always will prefer a review from someone that paid full price.
Interesting. Anyone spotted some games that are noticeably impacted?
d00d3n
Member
It is easy to check one title at a time by changing the filter settings. I haven't found a game that has been impacted in the sense that the overall % has been pushed down dramatically btw. The only games I have found with large discrepancies between keys and purchases are No Man's Sky and Arkham Knight, but the overall score is not shifted dramatically in any of these cases.
Having just messed this around a little bit I'd like to see a few tweaks to the system:
- There's no way to get "Recent Reviews" data that isn't under the default settings
- I hate having to hover over the rating to get the percentage, it makes some sense at the top of the page but there's enough free space in the filter box to display that info
- I wish there was a way to set default filters for my account and have the information at the top of the page and in the store search be filtered accordingly
Also in the future I'd like to see charts of the ratings over time with major patches marked but that's more relevant to the overall review system than this particular update.
- There's no way to get "Recent Reviews" data that isn't under the default settings
- I hate having to hover over the rating to get the percentage, it makes some sense at the top of the page but there's enough free space in the filter box to display that info
- I wish there was a way to set default filters for my account and have the information at the top of the page and in the store search be filtered accordingly
Also in the future I'd like to see charts of the ratings over time with major patches marked but that's more relevant to the overall review system than this particular update.
NeoRaider
Member
What's the point of writing a review if it's not included in the overall score??
Also i am getting exactly the same product from GMG or Humble Bundle for example, so i don't really get this. It's just stupid.
Why would only customers who bought the game directly on Steam have that "privilege" to be included in overall score?
Also i am getting exactly the same product from GMG or Humble Bundle for example, so i don't really get this. It's just stupid.
Why would only customers who bought the game directly on Steam have that "privilege" to be included in overall score?
Maybe if you read the OP...
PepsimanVsJoe
Member
That's fine.
However, just because I received a game for free doesn't mean that I'll give it a positive review. All a free key guarantees is a review, it could go either way, depending on the quality of the game (or at least how I perceived it). Valve's data is right about a lot of things, but I was always 100% dedicated to writing fair and thorough reviews.
I probably enjoy writing about games more than I do playing them, but if my contributions are "colored" by how much money I spend, then I guess there's no point in continuing. I'm not interested in bankrupting myself, just for a faint glimpse at legitimacy.
EDIT: Despite my complaints, overall this is a great move for developers as well as consumers.
grandwizard
Member
It's a bummer they had to do this, but something had to be done and I think this is a reasonable course of action. I'd rather have some of my reviews (via key buys offsite) not count in aggregate than have inaccurate scores all over the place due to shady review groups and developer interference.
RealityCheque
Banned
But now where will I go to get excellent dinner recipes?!
exmachina64
Banned
Serious question:
If I understand you correctly, you've written more than 240 detailed reviews on each game's store page for no compensation. Why? Is it simply out of passion?
I don't know what you do for employment, but if you have or could make some contacts, you might be able to write reviews for a living.
hobblygobbly
Member
Seems really detrimental for a lot of crowd-funded games - none of their backers can have any input on the review score.
Moral Panic
Member
I usually get steam keys for indies from their web site, so its a bit of a shame.
The score with those can be seen when you change the filter but not the main score displayed at the top. While kickstarter projects will be an issue now, this also does away with the inevitable bias some kickstarters can bring, as these are people already invested in a game, so it's interesting that these will be dependable for the first time, while also being visible via the filter
PepsimanVsJoe
Member
Passion. I have a full-time job that pays the bills.
Even though I've been writing about games for several years, I haven't been able to find any steady paying work. Most of my contacts have either dropped out entirely or they're doing something else.
I've received offers to write for a handful of websites. It's always the same story though. I have to write about the same games, publish half-a-dozen meaningless previews a month, and get paid next to nothing. I'd rather get paid nothing, but at least have the freedom to write about whatever I want.
Theanine3D
Member
It actually doesn't.
Developers who were truly trying to cheat the system can freely continue doing so. They're just a bit more inconvenienced now in doing so. They can still just send a Paypal payment (of the game's full pricetag) to a bought-out reviewer so that the bribed reviewer can manually buy the game through Steam for free.
Meanwhile, all people who were legitimately reviewing the game, positive or negative, are punished if they happened to have bought the game on itch, Humble, or elsewhere.
d00d3n
Member
That will be quite expensive though, given that Steam gets a share of the money as well.
PepsimanVsJoe
Member
Couldn't a developer generate something ridiculous like a 95% off coupon for their game, and then give it to the fraud-reviewer along with the cash to cover the difference? The game is still considered a purchase through Steam.
Most of these games that get boatloads of paid reviews are already dirt cheap anyway.
Most of these games that get boatloads of paid reviews are already dirt cheap anyway.
Yeah that is what I meant. They have to buy it through Steam (using whatever means, like a third party). At least it is more costly, but for big publishers it is still not going to be a problem. The only thing they would be loosing is VATs and the 30% Steam tax, the rest goes back to their own wallet. Pocket change; buying, say, 15000 copies will probably be worth it if it changes the rating.
Abilidebob
Member
If you do it for passion, what does it change for you if your review doesn't count for the % aggregation? It's not like your reviews can be summarized into Like or Don't Like right?
This really, really sucks. As usual Valve are using a sledgehammer to crack a nut, and a suspiciously anticompetitive sledgehammer at that.
They even state that they can detect review manipulation and plan to ban developers who do it... but then decide that's too much effort, much easier to just marginalise non-Steam purchases.
And that's the real bottom line: This is a small, subtle dig at non-Steam purchasers, but it's part of a worrying trend of Steam closing itself in towards becoming a fully walled garden.
They even state that they can detect review manipulation and plan to ban developers who do it... but then decide that's too much effort, much easier to just marginalise non-Steam purchases.
And that's the real bottom line: This is a small, subtle dig at non-Steam purchasers, but it's part of a worrying trend of Steam closing itself in towards becoming a fully walled garden.
John Kowalski
Banned
You know it's really hard to read the OP like that.
Theanine3D
Member
It won't be a problem at all for all those devs with RPGmaker games on Steam that cost $1 - $3. It's chump change.
Oh god, this is reeeeeeally bad for smaller indie developers. Like, my game's review count now went from 27 (93%) to 6 (83%, because one guy was like "game is awesome, but add Russian translation" and gave it a negative review), as most of them were from reviewers/blogs and also Humble Store.
Imagine a game with only a few reviews gets into something like Humble Bundle, receives hundreds of positive reviews that could help sell the game when the bundle is over, only that.... the Steam page simply won't count them.
Of course it won't really matter for big games and who cares about Indies. And as some of you have mentioned, this new system can be exploited just like the old one, if you have the money (or wait until a Steam sale).
Imagine a game with only a few reviews gets into something like Humble Bundle, receives hundreds of positive reviews that could help sell the game when the bundle is over, only that.... the Steam page simply won't count them.
Of course it won't really matter for big games and who cares about Indies. And as some of you have mentioned, this new system can be exploited just like the old one, if you have the money (or wait until a Steam sale).
PepsimanVsJoe
Member
I don't care about aggregation, but I like it when people read my reviews.
At least this new system doesn't remove key-activation reviews from the Community Hub.
Sure, but would you trust a review from a key activation? Valve wouldn't.
It'd be one thing if the purchase-type in the store page defaulted to "ALL", but it always defaults to Steam Purchasers.
It doesn't remove them from the review section either. You just change the filter to whatever you want with a single click.
Case and point:
Vastly different sample sizes, admittedly, but No Man's Sky's figures are interesting:
Right now someone out there is very thoroughly analysing all of this.
It doesn't matter if Valve choose one default or another. If you care about reviews, you should be checking out the reviews section of the page rather than the singular aggregate, and if curious, the scores per filter are easily viewable with a click / hover. If anything the ability to now see the difference proves more informative.
It removes a form of bias that can exist from keys given for free, kickstarters investors etc etc. Would I want to do this all the time - no. Would I like the option and the default to be the least source to have bias - yes
SerTapTap
Member
This is awful for a lot of reasons people don't seem to be considering:
This fix solves nothing but significantly harms certain classes of small devs that have nothing to do with the actual issue they're pretending to address. Complete idiocy.
- If your game has less than 50 reviews, it's in a drastically lower visibility tier
- Kickstarters initial/kickstarted sales are basically meaningless
- Reviews from reviewers are effectively meaningless because they get keys, just great.
- This is hilariously pointless considering Steam said they found automated reviews, just remove the automated reviews. They're not even addressing the actual issue.
- This is pretty anti-competetive to the likes of Humble Bundle/Itchio/etc.
- Steam would do FAR better to remove the curator system (LOTS of key beggers), hound after giveaway sites and have better info to train devs to not fall for those scams.
This fix solves nothing but significantly harms certain classes of small devs that have nothing to do with the actual issue they're pretending to address. Complete idiocy.
That's not a change.
Completely agree with the decision, while it doesn't completely solve exploitation of the system, since devs can still easily exploit the system by just buying keys through steam (using vpn's, possibly virtual machines, wallet cards and so on), it is an all around positive change, if absolutely nothing else due to the filtering options.
At the end of the day, the differences I have seen haven't been particularly relevant, after reading the RPS article on the subject, an article I strongly disagreed with due to beeing heavily biased and making unfair broad generalizations, I run the numbers on the 2 relevant games there, and others I found more relevant, and the % differences have been borderline insignificant.
RimWorld which was being sold for quite sometime outside of steam, and only relatively recently started selling on EA, saw a 2% difference.
Factorio a game in a similar position resulted in a 0% difference, even the most relevant games out of those in the RPS saw a 1% difference.
So with these differences if it ends up helping fighting devs trying to exploit the system, or at least makes it more expensive (which it does) it can only be a good thing.
lol.
At the end of the day, the differences I have seen haven't been particularly relevant, after reading the RPS article on the subject, an article I strongly disagreed with due to beeing heavily biased and making unfair broad generalizations, I run the numbers on the 2 relevant games there, and others I found more relevant, and the % differences have been borderline insignificant.
RimWorld which was being sold for quite sometime outside of steam, and only relatively recently started selling on EA, saw a 2% difference.
Factorio a game in a similar position resulted in a 0% difference, even the most relevant games out of those in the RPS saw a 1% difference.
So with these differences if it ends up helping fighting devs trying to exploit the system, or at least makes it more expensive (which it does) it can only be a good thing.
That's not a change.
lol.
All this seems to have accomplished is making things harder on indie devs. They even state it was like 160 games that were the culprit. That's a pretty small amount of games to have to worry over, and make this change to combat against.
Yeah, the system was fucked, but it's not exactly unfucked now and may harm developers and genuinely good games from getting attention.
Yeah, the system was fucked, but it's not exactly unfucked now and may harm developers and genuinely good games from getting attention.
Infinite_Daremo
Member
Whats funny is I cant submit a Steam Controller Config without having used it for over an hours time. But I can play a game for 20 seconds and declare it utter trash. But to be fair, limiting a games review potential on time is also bad because what is the game crashes on startup for a lot of people.
You guys realize that you can still write your reviews, and still have them show up both in your friend feeds and on the game's main page, right? Literally the only thing that changed is that the your thumbs up/thumbs down score won't be added to the aggregate score at the top if you got the key elsewhere.
That is literally the least important part of any review where you actually care about the text you're writing about the game. People can still see if you liked the game or not. They can still see what you wrote about it. They can still see if other users thought your review was useful and/or funny. All that stuff is exactly the same. The only difference is that the positive/negative score at the top of the page won't count your review.
If that's the only part you care about, then why do you bother writing text at all? The important part is the thumbs up or thumbs down, and beyond that you might as well just write "this game is good" or "this game is bad".
But go ahead, take it as a personal insult, I guess. That makes sense.
That is literally the least important part of any review where you actually care about the text you're writing about the game. People can still see if you liked the game or not. They can still see what you wrote about it. They can still see if other users thought your review was useful and/or funny. All that stuff is exactly the same. The only difference is that the positive/negative score at the top of the page won't count your review.
If that's the only part you care about, then why do you bother writing text at all? The important part is the thumbs up or thumbs down, and beyond that you might as well just write "this game is good" or "this game is bad".
But go ahead, take it as a personal insult, I guess. That makes sense.
I disagree with pretty much all of this but especially this concept of it being anti-competitive, you're talking about a service that lets developers generate an infinite number of game keys which they can sell or give away anywhere they want and Valve sees no return from, all they are doing it trying to ensure that the user review system is legitimate.
It may hurt some people, it may help some people, but I think Valve have been more than fair to any other storefront that wants to take a swing at them.
To the best of my knowledge that is not true, for people to see the written review they now have to click one button to show all reviews (or the steam key reviews), so by default the text reviews made by people with steam keys don't show unless the user reading specifically selects to show them, which to me makes no difference I will still click on that to read the reviews, but it is not just the score that changes.
Baron von Loathsome
Member
"Customers that received the game from a source outside of Steam (e.g. via a giveaway site, purchased from another digital or retail store, or received for testing purposes from the developer) will still be able to write a review of the game on Steam to share their experience. These reviews will still be visible on the store page, but they will no longer contribute to the score."
Just confirmed what I said, and I was correct, you now need to click 1 extra button to be able to see the reviews from people that activated the game with steam keys, you can check yourself using this game page http://store.steampowered.com/app/337720/ which has very few reviews, and a more or less 50/50 split between both types.
paperspace
Member
I don't see the problem?
You are for forgetting the very important part that your review won't be read anymore. Which, ya know, its the point of writing it? The majority of people will not change the default setting. And the default setting resets everytime you access a page.
Mobius and pet octopus
Member
People are under estimating the clarity this brings to the review system. I've seen enough gamers accuse developers of manipulating their score. Now you have a better idea whether this is true or not. Before it was just baseless accusations.
The review system is meant to help consumers first, not developers.
The review system is meant to help consumers first, not developers.