I think your MCC date is wrong
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
From Stage to Screen: How fast do conference games release? (PS4, Xbox One)
- Thread starter Liabe Brave
- Start date
Michael Armbrust
Member
Halo 5 should be E3 2013 not 2014 for reveal, right? Name wasn't known but that's when they announced the game and 60fps.
I LOVE data-driven posts like this. Bravo!
Nightengale
Member
I had it fixed in the other table, but forgot to fix in original. Done.
I guess this is where what constitutes as teasing/official announcement can differ from people to people.
Michael Armbrust
Member
Dr. Kitty Muffins
Member
I like the idea of being able to preorder on PSN just after the game has been shown.
Melchiah
Member
If only we could still pre-order digitally without paying months in advance like we used to. I miss the time, when the pre-orders were charged a couple of days before the release, just before the preloading began. Too bad it was changed, possibly because people abused the system to get beta access and whatnot.
Liabe Brave
Member
Great, thanks! It's good to know that we reached basically the same end; I think that helps other people trust the analysis.
For anyone that's looking at Nightengale's data, please keep in mind that he collected all first-party games, regardless of whether they appeared in a press conference or not. Evidently this doesn't materially affect the results, but it may cause some mismatch in the data between us.
I actually counted the E3 2013 showing as being TMCC alone. The CG they showed isn't part of that, but otherwise everything they say applies to TMCC, not Halo 5. And with hindsight, it's clear Phil Spencer and Bonnie Ross are going out of their way to not say "Halo 5". I think they knew they weren't talking about its release, but wanted to be vague about it.
Which is why I don't refer to "announcements" at conferences, but just "appearances", or whether a game was "presented". It eliminates most of the opportunity for confusion (though not all, as the Halo discussion above shows).
I have one other bit of data to post. I've kept it until last because it isn't very germane to the question about release schedules. But I had to generate it on my way to that analysis, so I figure I might as well share. It's the breakdown of release numbers, split by platform, exclusivity, and type.
As with my previous posts and as said in the graphic, "indie" here is an assessment of relative production value, not of actual publishing duties. In addition, exclusivity is "soft"--that is, whether the directly competing console has it. So a game on PS3 and PS4 but not Xbox One would count as a PS4 exclusive; a game on Xbox One and PC but not PS4 would count as an Xbox One exclusive.
Let me know if I've posted anything unclear, or if you have other questions about the data.
hydrophilic attack
Member
I have one other bit of data to post. I've kept it until last because it isn't very germane to the question about release schedules. But I had to generate it on my way to that analysis, so I figure I might as well share. It's the breakdown of release numbers, split by platform, exclusivity, and type.
Interesting. I missed the indie games during Sony's 2016 conferences
Huh. Not what I was expecting either as a result, especially with Sony having shown God of War, Days Gone, Spider-Man, and Detroit: Become Human over a year ago IIRC with none of them launching yet. And of course the albatross that is Deep Down. Then again, Microsoft's score has to be weighed down by things like Cuphead and Crackdown 3. Hell, didn't they show Crackdown 3 back in mid-2014?
Either way, good analysis and number crunching.
OP, can you share with us the Top 5 or 10 longest in-dev games for each platform this gen? I'm curious what those would be.
Either way, good analysis and number crunching.
OP, can you share with us the Top 5 or 10 longest in-dev games for each platform this gen? I'm curious what those would be.
Liabe Brave
Member
Keep in mind that I mean indie as in "relatively lower production value", not per publisher. This therefore includes highly-polished experiences that are very short, like Batman Arkham VR. Most such titles come from PlayStation Experience (Batman was the only one shown at E3).
Of that list, only Detroit is older than a year (first shown at the very end of October 2015). The other three were first shown at E3 2016, which is a little less than a year ago.
Sure.
Code:
[U]MICROSOFT[/U]
[I]Below [/I] 47 months (in dev)
[I]Aztez[/I] 34.5 months (in dev)
[I]Crackdown 3[/I] 34.5 months (in dev)
[I]Cuphead [/I] 34.5 months (in dev)
[I]Quantum Break[/I] 34.5 months (released)
[U]SONY[/U]
[I]deep down[/I] 50.5 months (in dev)
[I]Kingdom Hearts III[/I] 47 months (in dev)
[I]Ray's the Dead[/I] 47 months (in dev)
[I]Guns of Icarus Online[/I] 44.5 months (in dev)
[I]Rime[/I] 44.5 months (in dev)
Liabe Brave
Member
This is incorrect. Here is a direct quote from the conference.
Bonnie Ross said:
That is very clearly an announcement of a single product. Everything that's said there--and what goes unsaid, the words "Halo 5" or "sequel" or "continuation"--indicates to me that Ms. Ross was talking about the Master Chief Collection, but didn't want to clarify that the date shown was for a remaster, and the next numbered game was still more than two years away. If you'd rather believe she was indeed speaking of Halo 5 (and dancing around the name for some reason), which eventually got delayed a year and was replaced in the schedule by a quickly thrown-together MCC, that's also possible.
But the segment is very much about one title, not a whole series (though at least one more product is promised to follow).
Great idea for a thread, and I really appreciate your data collection and methodology, great work OP
I really do have to say though that I don't think it's fair to count 3rd party titles against Sony or Microsoft. I can see counting Below since that is published by MS, but Deep Down is more than likely out of Sony's hands completely (and Capcom has more than proved they have an issue releasing software this gen)
I really do have to say though that I don't think it's fair to count 3rd party titles against Sony or Microsoft. I can see counting Below since that is published by MS, but Deep Down is more than likely out of Sony's hands completely (and Capcom has more than proved they have an issue releasing software this gen)
Liabe Brave
Member
I'm not counting anything "against" anyone; it's just data. Yes, the PS4/Xbox One comparison is interesting because it runs counter to the usual interpretation. But that's not what I set out to do--I fully expected Sony to take much longer to release games. I collected the data because I was interested in the overall scheduling of reveal versus release. (If you're interested in just first party stuff, though, I did post those numbers earlier in the thread. There's full details for non-indie first party, and just the averages for all first party.)
To me, the most intriguing results haven't been directly related to Microsoft/Sony but to them as a group: that games on average only take a little over a year to come out, that Japanese developers are in general much better than western ones at sticking to their marketing leadup-to-release timing, and that AAA studios in general are faster than indies in general, not slower. Those also aren't usual topics of discussion, and these possibly-novel facts probably have more impact on the industry. They definitely affect the way I'll assess lengthy dev time in the future.
Brad Grenz
Member
I'm not sure a "hindsight is 20/20" take should apply in this case. They let everyone spend a year believing that was a Halo 5 teaser only to pull the rug out at the next E3.
Nightengale
Member
The perception isn't entirely off, per say.
My list has 34 non-remaster retails games announced by Sony so far, 10 of which are "in the making.
Of said 24 games, I could easily take off 6-8 games like Knack, Driveclub, MLB, LBP3 that aren't popular among GAFer's primary taste in games. It's not an uncommon thing that things we don't like aren't part of our key consideration when viewing a company portfolio.
When that happens, the number of "core" games released vs unreleased is not that big. And the unreleased games are also more notable in stature and hype, so there is the mental weight attached to the unreleased games vs the released ones.
Liabe Brave
Member
I think it's much more accurate to say that the perception is entirely off, but for understandable reasons. Yes, you can ignore games of lesser stature, and games without much cachet, and games that come out reliably, and end up with a subset that matches perception. But if that's a valid process, you can use it to say that any perception isn't far off. (That "Microsoft doesn't care about games outside their core franchises", for example. It takes very selective memory to call this even remotely true.)
This all applies only to generalized statements, of course. There are more specific ones--"Sony has more games with long dev times", etc.--that are quite true, but those don't exist in a vacuum. They need the context of the library around them, Sony also having more games with short dev times and so on.
This is basically another in the long line of lessons that real data is noisy, unpredictable, and not conducive to simple generalizations.
DriftingSpirit
Member
The only thing far off from last year will be Death Stranding. I guess it depends on what we mean by "far off" though. For me, I think Death Stranding won't be coming in 2018. Everything else will.
Liabe Brave
Member
Which is a great example of my theory about this idea being caused by selective memory. Because what you're actually saying is, "The stuff they showed last year that comes to my mind is a long way off."
By my guidelines, Sony presented 11 games last E3. Of those, 7 are already released.
I bet that proportion is surprising to a lot of people, who are concentrated on specific games that have languished.
azertydu91
Hard to Kill
Yeah that really put in perspective the whole selective memory thing it's like judging a whole lineup by only a ridiculously small fraction.
But they're still basically quite close in the average release time frame.
Nice work OP. Makes for very interesting reading.
So the opposite of what some of the MS diehards on Gaf constantly post is actually the case then? Quelle surprise.
So the opposite of what some of the MS diehards on Gaf constantly post is actually the case then? Quelle surprise.
Fascinating reading OP. I really appreciate you compiling all this data. Also think that it's interesting that a t-test on the modal interval is insignificant. That really cemented to me that there's really not as much of a difference as people think.
Here's another question: Are you thinking of continuing data collection as this generation continues? It would be interesting to see the numbers once it's all said and done.
Here's another question: Are you thinking of continuing data collection as this generation continues? It would be interesting to see the numbers once it's all said and done.
Based on the trend so far, I can't see the level of significance changing much.
Liabe Brave
Member
It's not as clear-cut as you seem to want. Yes, general statements about the overall approach of Microsoft versus Sony aren't warranted by the data. "Sony take longer to release games", "Microsoft only announces stuff coming out in the next 12 months", etc. are wrong.
But the fact that Sony shows and releases many more games, period, makes absolute number comparisons sometimes still valid. For example, "Sony's longest-interval games have been waiting more months than Microsoft's longest", or "Sony has more games that have taken over two years to come out" are both true statements. It's just that you usually don't see folks acknowledge that "Sony's shortest-interval games came out faster than Microsoft's", or "Sony has more games that have taken less than two years to come out" are also true.
People are prone to recall specifics, where Sony may indeed take longer than Microsoft, rather than the overall shape of the population. It doesn't make the mistaken generalizations accurate, but it seems obvious why they arose.
Since I'll almost certainly watch the conferences anyway, I don't see any reason to stop compiling the data. But I don't think it'll alter meaningfully at a very rapid clip (part of the reason I thought to do this now instead of earlier is because there was a nice buildup of info). So I don't think it'd make sense to post a new summary until at least a year has passed. It's likely everybody reading it would just be trying to overfit a predictive curve.
Dude, reading you feels so good somehow, it feels like I'm reading rationality itself explaining us gaming related stuff!
Insane work on this one, I enjoyed everything I read!
I dont think you are wrong. Especially since last year MS only shows games that are close to release.
Liabe Brave
Member
I posted the Sony list earlier in the thread. Here's the games shown at Microsoft conferences but later canceled:
Shangheist (Black Tusk spy action, dropped when studio repurposed to Gears of War)
Fable Legends (Lionhead co-op versus, canceled and studio eliminated)
Hellraid (Techland first-person fantasy, put "on hold" in 2015 to focus on Dying Light)
Phantom Dust (Darkside card action reboot, put "on hold" in 2015, studio dissolved)
Scalebound (Platinum co-op dragon action, canceled this January)
Ion (Rocketwerkz/Improbable SF survival, collaboration ended in 2016)
As it turns out, he was wrong, and you're mistaken to think otherwise. If we look solely at 2016, here's the data for what the platform holders showed off:
Code:
Shown Released In development Avg. Interval Avg. Interval (Released games only)
Microsoft 20 7 (35%) 13 (65%) 10.2 months 6.1 months
Sony 38 15 (39.5%) 23 (60.5%) 5.9 months 4.1 monthsThe overall patterns for the generation still held last year: Sony presented roughly twice as many games, and released roughly the same proportion within a short time period. Note that while I included the stage-to-screen interval here, it's even less accurate than usual since the majority of games shown in 2016 haven't released yet; both platforms' average will rise by a notable amount once all games are out. Sony's number is particularly attenuated, since the December PlayStation Experience means some of their games were shown much more recently.
For those reasons, I've also included the average interval just for the titles that have come out. While this will just exacerbate the systematic underestimation for both sides, it should eliminate the distorting effects of presentation scheduling, and even out the playing field so the platforms are more comparable. And we do see the relative gap shrink significantly...but Sony's average interval doesn't become higher.
funkystudent
Member
Wasnt Below announced the year before Crackdown 3?
DriftingSpirit
Member
Yeah, Microsoft's 2013 E3.
Cheers!
Liabe Brave
Member
Yes, this is why in the OP it's listed as the maximum interval game. Crackdown 3 is shown in the same spot in post #18 because there, I'm listing only non-indies (in the sense of production value, not publisher). Sony has deep down as max interval in both posts since it applies in both. The indie game with the longest interval for Sony is Ray's the Dead, 47 months just like Below.
No problem!
the difference between announcement and release is small between companies; maybe Microsoft just prefers to release more games in, say, december, while sony prefers november? or maybe just microsoft is reacting to sony release dates and pushing stuff a little later?
p good job anyway
p good job anyway
Liabe Brave
Member
These conjectures, even if true, wouldn't change the conclusions shown by the data, though. They're attempts to answer why Microsoft doesn't have a shorter release interval, despite general belief to the contrary. But if we're in agreement about the truth of the unexpected pattern itself, it is interesting to take a shot at an explanation.
So I checked the release months of all presented games that have come out (nothing canceled or still in development). Compared to Sony, Microsoft has released about half as many previously-showcased games each year (and in 2014 they only put out about a third), so comparisons by absolute number are disjunct. Instead, I graphed releases as a percentage of total output. Since 2013 and 2017 are incomplete years, I only included 2014-2016. Here's the results:
Once again, there's info here that runs counter to public belief. The common perception--including my own, before I ran this--is that Microsoft concentrates their releases much more heavily in the fall/holidays, and Sony releases more in the earlier "preseason". This may be true for first-party efforts or for all games (I didn't check, so maybe not!), but is definitely not accurate for the set of games that have been showcased at press conferences. While Sony does concentrate more such titles in the first half of the year--45.5% versus Microsoft's 38.9%--their heaviest months are still during the holidays. And Microsoft isn't as idle during the start of the year as perceived.
Other points of interest shown here:
- Release timing across platforms is strongly bimodal, with peaks in calendar Q1 and Q4
- Neither platform likes to release in December (only 6 games between them)
- October is Sony's highest month, due to last year's VR launch
- Within H1 and H2 separately, Microsoft is balanced earlier than Sony; note February rather than March, and the higher ramp starting in late summer versus low September
Unfortunately, I don't know that we'll ever be able to explain the causes of this. It seems to me that most plausible candidates are internal to the business: efficiency of studios, strategic selection of conference elements, development delay management, etc. We simply don't have access to the details of these things to do a proper multivariate analysis, and we probably never will. (Though perhaps there'll be tell-all books at some future point!)
I'm happy to answer any questions or take a stab at any analysis ideas people do have, though. Just because we can't be certain doesn't make it less fun to speculate.
Third parties being announced/debuted on a MS/Sony stage are there at least partly because MS/Sony see it as adding value to their brand/marketing. And it is a responsibility of MS/Sony to understand where those third party games are in their development cycle and be comfortable bringing them on stage.
So it is absolutely fine for them to be included here.
Psycho_Mantis
Banned
Great thread OP.
Some issues I take with the application of the methodology...
1. You state the following in the OP: "For these purposes, "shown" means displayed footage with an associated title."...yet you later seemed to suggest that you abandoned this when deciding when to count Halo: TMCC as being announced.
2. You are not actually addressing the kind of research question necessary to speak to the general impression you sought to clarify. In other words, if you sought to understand whether the general impression in the community, which suggests Sony has a longer window for announce-->release, then you should not be counting 3rd party titles at all. 3rd party titles do not generally contribute in any way to that impression so why would you include them in trying to address said impression?
3. Starting at E3 2013 would seem to strongly favor one competitor here over the other since it somewhat arbitrarily restricts the analysis only to games announced for current gen. Of course, there are high profile games form last gen that got announced and had to get revived on PS4, like The Last Guardian.
4. Your methods skew the data to favor the platform with more titles recently announced. As those titles get announced their time to release starts at zero and seems like it should drastically drag down averages. You note this issue in your OP but seem to act as if it is no germane to your conclusions somehow. But...it totally distorts the context for how you are interpreting the data if you count it. In other words, the data is now reflecting information that is suddenly relevant to a totally different question than you set out to look at, yet your interpretation did not change.
5. Refusing to share your data makes zero sense. That is borderline indefensible given the stated purpose of your 'research' efforts. That is not how serious research is done. I know, 'vidya games' or whatever but when ya put so much effort into the OP and follow up replies it becomes hard to imagine a rationale for refusing to share the raw data.
6. I know some of these issues are already mentioned and discussed beyond the OP, but the fact you did not seek to address them a priori makes me even more dubious as to your actual application of an appropriate methodology.
7. I checked the 18 titles you said you used in your Xbox One analysis for only first/second party titles, excluding Crackdown 3 since that is not released yet. Using the dates that I found online for when each was announced/released I got pretty significantly different numbers than you did. I can't see what you did in terms of your actual calculations since you refused to post your data, but I got 10.76/12.12 "months" (aka 30 days) for the 18 Xbox One games depending on how we count Ryse.
Further, if I include Dead Rising 4 (why did you leave that out?) I arrive at 10.5/11.78 depending again on how we count Ryse. When I add Zoo Tycoon we get 10.25/11.47.
8. Here, this is what you said was your research question:
But you did not provide any contextual stats analysis at all, making the notion that you were researching probabilities about future release intervals and/or cancellations and whatnot nonsensical. You don't use small samples and frequentist stats for that kind of analysis, ya look towards Bayesian models instead.
What your data tells you is merely how likely it is that had you made a list of all announced titles thus far and closed your eyes and chosen one at random, how likely it would be that the title you chose was released as of today. That is very different than your stated questions quoted above.
1. You state the following in the OP: "For these purposes, "shown" means displayed footage with an associated title."...yet you later seemed to suggest that you abandoned this when deciding when to count Halo: TMCC as being announced.
2. You are not actually addressing the kind of research question necessary to speak to the general impression you sought to clarify. In other words, if you sought to understand whether the general impression in the community, which suggests Sony has a longer window for announce-->release, then you should not be counting 3rd party titles at all. 3rd party titles do not generally contribute in any way to that impression so why would you include them in trying to address said impression?
3. Starting at E3 2013 would seem to strongly favor one competitor here over the other since it somewhat arbitrarily restricts the analysis only to games announced for current gen. Of course, there are high profile games form last gen that got announced and had to get revived on PS4, like The Last Guardian.
4. Your methods skew the data to favor the platform with more titles recently announced. As those titles get announced their time to release starts at zero and seems like it should drastically drag down averages. You note this issue in your OP but seem to act as if it is no germane to your conclusions somehow. But...it totally distorts the context for how you are interpreting the data if you count it. In other words, the data is now reflecting information that is suddenly relevant to a totally different question than you set out to look at, yet your interpretation did not change.
5. Refusing to share your data makes zero sense. That is borderline indefensible given the stated purpose of your 'research' efforts. That is not how serious research is done. I know, 'vidya games' or whatever but when ya put so much effort into the OP and follow up replies it becomes hard to imagine a rationale for refusing to share the raw data.
6. I know some of these issues are already mentioned and discussed beyond the OP, but the fact you did not seek to address them a priori makes me even more dubious as to your actual application of an appropriate methodology.
7. I checked the 18 titles you said you used in your Xbox One analysis for only first/second party titles, excluding Crackdown 3 since that is not released yet. Using the dates that I found online for when each was announced/released I got pretty significantly different numbers than you did. I can't see what you did in terms of your actual calculations since you refused to post your data, but I got 10.76/12.12 "months" (aka 30 days) for the 18 Xbox One games depending on how we count Ryse.
Further, if I include Dead Rising 4 (why did you leave that out?) I arrive at 10.5/11.78 depending again on how we count Ryse. When I add Zoo Tycoon we get 10.25/11.47.
8. Here, this is what you said was your research question:
But you did not provide any contextual stats analysis at all, making the notion that you were researching probabilities about future release intervals and/or cancellations and whatnot nonsensical. You don't use small samples and frequentist stats for that kind of analysis, ya look towards Bayesian models instead.
What your data tells you is merely how likely it is that had you made a list of all announced titles thus far and closed your eyes and chosen one at random, how likely it would be that the title you chose was released as of today. That is very different than your stated questions quoted above.
DriftingSpirit
Member
Nah, it doesn't matter. People already have their mind made up and they believe what they want to believe. As it has been pointed out, the grandiose nature of Sony's side (FF7R, Shenmue, TLG) just make the perception worse for them.
backbreaker65
Banned
He's not the only one that came to that conclusion, you did no research, but have a problem with his methodology... LMAO, you're not refuting. Here's someone else that arrived at the same conclusion.
https://www.reddit.com/r/PS4/comments/6ga4ru/gaming_press_conference_releases_a_study_on_how/
BRocknRolla
Member
I absolutely love seeing people shut down with real data. It's a beautiful thing.
That set of data also does not address the question posed here by the OP. And yes, I am refuting the notion that the methods used here somehow are germane to addressing the research questions the OP specified. The methodology simply does not stand up to scrutiny.
And fyi, that data has some major errors in it. Last I checked it too more than 3 days for The Last Guardian to release.
That also counts literal re-releases, like some games showing up literally the day they are announced. That list also claims FFXV was only 77 days between announcement and release...lol
That data is all sorts of bogus actually. The number for all sorts of games are all screwed up. It seems to be counting the days from the last time it was shown at a conference to the day of release instead of looking at the day the game was announced or first presented.