charlequin
Banned
donny2112 said:
Yeah, I'm pretty clear on that. I just wanted to make a point about dialmydrive's claim that I'm spreading "misinformation" about what subscribers are allowed to do with the data. :lol
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
NPD To Produce First Digital Download Chart in US? Wants to stop the misinformation
- Thread starter FrankT
- Start date
Yeah, I'm pretty clear on that. I just wanted to make a point about dialmydrive's claim that I'm spreading "misinformation" about what subscribers are allowed to do with the data. :lol
dialmydrive
Member
:lol :lol :lolcharlequin said:
elrechazado
Banned
AstroLad said:
lol @ you.
LizardKing
Member
a lot of NPD hate in this thread. true you can get their info (or at least sales data) for free in other countries. But, I thought they were pretty reasonable back when gaf used to get the leaked npd data (the full amount) illegally. Instead of shutting it all down they gave us the top ten every month for free. I forgot lots of people here might not know that
I remember reading somewhere on GAF that NPD use the same Walmart multiplier for every game, no matter the title or platform.
Is that true? Because that seems pretty dumb to me.
I wish the industry would just get together and fund their own tracking service. NPD seems like crap to me. If they're entire business model is based on simple raw numbers for titles, they need to get some better analysts. That's what they should be doing. Look at trends. Breaking down things by genre, price points, stores, hell even areas. I just can't see why the raw numbers themselves are considered so valuable. It should just be an automated service at point-of-sale. Media-Create and Enterbrain have no trouble releasing weekly lists with the numbers. I can find out the gross revenue of pretty much every movie released in the US on the Monday after the weekend. Box Office Mojo don't charge me crap.
We should start GAF Tracking Inc.
Is that true? Because that seems pretty dumb to me.
I wish the industry would just get together and fund their own tracking service. NPD seems like crap to me. If they're entire business model is based on simple raw numbers for titles, they need to get some better analysts. That's what they should be doing. Look at trends. Breaking down things by genre, price points, stores, hell even areas. I just can't see why the raw numbers themselves are considered so valuable. It should just be an automated service at point-of-sale. Media-Create and Enterbrain have no trouble releasing weekly lists with the numbers. I can find out the gross revenue of pretty much every movie released in the US on the Monday after the weekend. Box Office Mojo don't charge me crap.
We should start GAF Tracking Inc.
elrechazado
Banned
so?apujanata said:
This information would be useful. It would let gamers know how a game has done, so they don't skip a game because it's a 'bomb' when it is not in fact a bomb. This will only encourage publishers to push for first-week sales though, so that their game will pop high on the charts and they can then act as a form of positive press.
alicemudgarden
Banned
dialmydrive said:
Mr David Riley (NPD)! It has been a while - good to see you are still as inventive as ever when it comes to fake usernames (an anagram of David.M.Riley) and supporting NPD without actually letting on that you work for them! You've lost none of your talent. Shame we didn't get to meet up at Pachter's E3 party, would have been good to personally shake your hand.
It must be galling to Google your own name, click through to a site like this and see your organisation being ripped apart despite all the hard work you do but know that you can't really defend yourself without giving it away (although I know you too well). Bravo! :lol
Right, on to the issue at hand - FADE is NOT VG Chartz, let me make that totally clear. VG Chartz bought a small stake in FADE two months ago, realising the potential of the company and helped to get it set up and promote it. FADE is a totally seperate entity in terms of data collection, staff and methodology and a totally seperate business.
Now, I'm as interested as everyone else to see exactly how NPD intend to collect this digital download data. Fact is, that the only way to get hard data is from either the publishers / developers or from the vendors - i.e Steam, Microsoft, Sony, Nintendo. Now, the contracts between the vendors and publishers is such that the vendors are not allowed to release any data to an outside source. So this leaves NPD in the situation where they can't really do any more than FADE is - collecting data directly from publishers who wish to participate (a great start but lots of gaps), partnering with large websites who specialise in supporting the various dd services and using user review data (assuming that a certain proportion of buyers will submit a user review), leaderboard monitoring, gamercard / psnid / nintendo channel monitoring, a large consumer panel and other in-development methods to produce the monthly charts and fill the gaps where data isn't directly shared by developers. FADE has been doing this since June and is already being adpoted by many in the industry as the official agency to cover these markets. It is pretty clear that this has forced NPD to come forward with this "noble" announcement
Moving on to the wider subject of VG Chartz - it still amuses me that so many on here believe that a site that is now bigger than GAF is run by some guy in his bedroom making up random numbers! I mean really, how pathetic?! Go check the VG Chartz US retail preview for the last 3 months (posted about a week before NPD) and the final NPD data - aside from a few differences (some of which I'll come onto) then the data is generally very close. For two totally independent data collection agencies (one mainly using retail data, one mainly consumer data) to post figures that are consistently within 10% month on month is impressive to me - but evidently not to most of you. I'm sure many of the GAF sheep can't even remember why vgc was banned in the first place! It might be hard to believe that anyone other than an organisation that has been going for 30 years could produce accurate weekly videogame charts but everyone has to start somewhere. VG Chartz has come a long way in 3 years.
legend166 said:
Not entirely true - from my understanding they use one factor per platform. So all Xbox360 games will be scaled by the same factor, Harry Potter (50% of sales in reality at Walmart) and Resident Evil 5 (20% of sales in reality from Walmart) alike. Some may call their "effective Walmart figures" nothing more than educated guesstimation (and not even that educated) - although using that kind of data gets you banned on this site doesn't it? ;-)
dialmydrive said:
Haha - go ahead and stop releasing data to the public then! It would have nothing but a positive effect on VGC, public NPD data is nothing but a thorn in the side. Publically releasing the top ten games for the month and the hardware sales in the US must really help VGC produce worldwide weekly charts of around 4000 games - I don't think. I guess the recent preorder data that VGC has started posting is "stolen" from NPD as well? Oh and FYI - VGC does provide dollar sales info but not publically.
Also, David, do you really want to go on record saying that the investor community is made up of dumb, gullible idiots? Not the best way to do business and make friends - but then that is why so many are starting to see VGC as a viable alternative to NPD - I think a lot of it is personal. Stunts like this won't help your reputation.
DD info is really centralized with almost no human error involved. Even well distributed PC games only go through a few different distributors, ten to twenty at most. All sales are recorded electronically.
What I'm saying is that the NPD would have to offer quite a bit to the major distributors to get them to talk considering there is no work involved in it. Quite the opposite of the hardware/retail software market.
What I'm saying is that the NPD would have to offer quite a bit to the major distributors to get them to talk considering there is no work involved in it. Quite the opposite of the hardware/retail software market.
Man God said:
Not so.
DD sales tracking will target the developer rather than the publisher.
The developer is informed exactly how much of their product has sold digitally and it is their best interests to divulge that information back to centralised tracking mechanisms for 2 reasons:
(i) if sales are good it offers solid reinforcement to the market of their potential and worth. This will affect future contract negotiations both with current publishers and potential new ones. Crucial for multiplatform development teams.
(ii) if sales are bad it highlights to competing publisher platforms a potential target for "crossing the divide" to develop exclusively for another side.
The only negative comes with continual bad sales figures which does denigrate their market value. If that happens they probably should pack the game up and head home anyway.
Isn't the developer already paid per game sold on the service anyways? They should have a great deal of info on exact game sales.
No, this news would only really be useful to investors and journalists, unlike retail sales. Accurate projections of those sales are worth quite a bit to a lot more people in the line. Investors, hardware, software, middlemen, journalists.
No, this news would only really be useful to investors and journalists, unlike retail sales. Accurate projections of those sales are worth quite a bit to a lot more people in the line. Investors, hardware, software, middlemen, journalists.
alicemudgarden
Banned
Well actually, overall market data is useful to all the developers who are currently in or hoping to enter the market.
So if I'm developing a small-budget puzzle game do I release it on iphone, DSiWare, XBLA, Steam? Which market is biggest, which would suit my game best?
Only by having data for the whole market can you make these decisions in an educated way. The issue is that to produce the data simply be talking to all the developers and aggregating the individual figures isn't going to work - many developers will have reasons for not participating so you would be left with loads of gaps in the data.
So if I'm developing a small-budget puzzle game do I release it on iphone, DSiWare, XBLA, Steam? Which market is biggest, which would suit my game best?
Only by having data for the whole market can you make these decisions in an educated way. The issue is that to produce the data simply be talking to all the developers and aggregating the individual figures isn't going to work - many developers will have reasons for not participating so you would be left with loads of gaps in the data.
dialmydrive
Member
Holy shit @ oioio. If anyone challenges him they're automatically David Riley. Anogram <sic> my ass. The word is that Fade is a vgczzz company. Denying it and saying you bought a stake in it is another lie.
Why is it that everyone who takes you to task is an automatic NPD or gfk employee, or whatever?
You are the most paranoid, ignorant, sad person in the world.
:lol
Why is it that everyone who takes you to task is an automatic NPD or gfk employee, or whatever?
You are the most paranoid, ignorant, sad person in the world.
:lol
alicemudgarden
Banned
David, GFK and ChartTrack have no issue with that VG Chartz does and vice-versa. They don't have employees sign up to forums to badmouth competitors. Only you sink that low ;-) It was obvious from your first post that you'd be up to your old tricks again!
dialmydrive
Member
David :lol
Anyone have an old GAF account they no longer want? (it's the sales site that's banned)
Hmmmm, interesting.
Hmmmm, interesting.
dialmydrive
Member
Taurus said:
OMFG I have accomplished much on this day. I am honored.
Segata Sanshiro
Member
Disregarding a "source" that can't even stick to basic rules of statistics isn't being a sheep, it's being a responsible consumer of data.
ioi and his bullshit were banned because he was presenting his data (at the time, just "borrowed" numbers from other trackers with some calculations made to them according to his own whimsy) as some kind of reliable data. It may interest Mr. Alicemudgarden (who may be ioi, may be square2005, may be some other tool) to know that I do compare ioi's pre-order data with the real numbers fairly often, and the size of the difference is sometimes absolutely fucking absurd.
Oh, I'm sure he's gotten a bit better at his little guessing game. Calling around to more stores, snarfing numbers from here and there when he can, but at the end of the day, ioi's numbers generally don't match NPD numbers or Media-Create numbers or Famitsu numbers when they release, until a few hours later when - hey whaddya know - they change!
You want to front for ioi, sir, that's fine - expect to be treated with the same credibility we give him.
ioi and his bullshit were banned because he was presenting his data (at the time, just "borrowed" numbers from other trackers with some calculations made to them according to his own whimsy) as some kind of reliable data. It may interest Mr. Alicemudgarden (who may be ioi, may be square2005, may be some other tool) to know that I do compare ioi's pre-order data with the real numbers fairly often, and the size of the difference is sometimes absolutely fucking absurd.
Oh, I'm sure he's gotten a bit better at his little guessing game. Calling around to more stores, snarfing numbers from here and there when he can, but at the end of the day, ioi's numbers generally don't match NPD numbers or Media-Create numbers or Famitsu numbers when they release, until a few hours later when - hey whaddya know - they change!
You want to front for ioi, sir, that's fine - expect to be treated with the same credibility we give him.
alicemudgarden
Banned
dialmydrive said:
What, getting me to post on GAF for a couple of hours before the inevitable ban?
Congratulations ;-)
Maybe next time we have this debate for real, face to face, in a neutral setting.
Segata Sanshiro
Member
Oh, nevermind, it is ioi.
So, when ioi says that:
"FADE is a totally seperate entity in terms of data collection, staff and methodology and a totally seperate business."
...this would be why ********' WiiWare charts for July:
http://news.********.com/news.php?id=4790
...are now FADE LLC's WiiWare charts for August:
http://news.********.com/news.php?id=5133
?
(Incidentally, I think these charts may be dangerously inaccurate to people thinking about making WiiWare games, especially regards median figures, which are MUCH MORE IMPORTANT than Top 10s for someone wanting to make a business -- and could actually harm their planning. This is what particularly upsets me.)
Full disclosure before I get accused of being David Riley - as a quick Google would probably reveal, I run Gamasutra and wrote the original article looking at ******** and accuracy for GameSetWatch:
http://tinyurl.com/5ap37q
I'd also been talking to some of the more accurate ******** folks (particularly the folks who did the XBLA ******** countdown, which is now, uhh, the Fade LLC XBLA countdown) about using the charts in some way or partnering in a business way.
But it's becoming increasingly clear (esp. after a few arguments with them) that they are willing to put out pretty abstractly interpolated numbers, with significantly varying degrees of accuracy, and then retreat to 'all charts are inaccurate - deal with it' rhetoric.
See the iPhone charts, which already got some press:
http://toucharcade.com/2009/09/08/top-iphone-game-sales-numbers-revealed-no-not-really/
TouchArcade themselves commented: "We contacted a number of the developers on the list, and so far, all of them report the numbers are well off target. Ethan Nicholas reports the sales of iShoot [$1.99] to be at 550,000 copies total so far again well off the 843,000 listed above. Illusion Labs (TouchGrind [$4.99]) and MikaMobile (Zombieville [$1.99]) have also responded stating their internal numbers are also well off Fade's. Also absent from the list are Flick Fishing which has sold over 1 million copies and Crash Bandicoot which topped Apple's own list of all-time top app sales."
It was a little bit better when at least the stats were out there and weren't being charged for. But if you're going to pay to get median WiiWare or iPhone numbers which then make you go in the wrong direction with your business, that's pretty galling.
Since VGC are often the only exact numbers out there, they have to be taken seriously as at least something to rebut. And in researching a GDC talk I did this year on digital download numbers (in which I cited carefully and gave pretty broad ranges, which is all you can do at this point), I looked at both XBLA and WiiWare charts from VGC - which are now the Fade LLC charts. The slides are up and you can probably work out what I said from them:
http://www.slideshare.net/simoniker/independent-games-sales-stats-101
(Honestly, I don't think it's easy for NPD to do a good job of most digital charts, either, given lack of reporting publishers and NDAs. So in this case, I'm not even sticking up for NPD over anyone else. If you look around you'll see a lot of people are making noises about digital charts, including Chart Track, but nobody is really getting it together, because there's no compelling commercial reason for platform holders and distribution points to give this info up.)
"FADE is a totally seperate entity in terms of data collection, staff and methodology and a totally seperate business."
...this would be why ********' WiiWare charts for July:
http://news.********.com/news.php?id=4790
...are now FADE LLC's WiiWare charts for August:
http://news.********.com/news.php?id=5133
?
(Incidentally, I think these charts may be dangerously inaccurate to people thinking about making WiiWare games, especially regards median figures, which are MUCH MORE IMPORTANT than Top 10s for someone wanting to make a business -- and could actually harm their planning. This is what particularly upsets me.)
Full disclosure before I get accused of being David Riley - as a quick Google would probably reveal, I run Gamasutra and wrote the original article looking at ******** and accuracy for GameSetWatch:
http://tinyurl.com/5ap37q
I'd also been talking to some of the more accurate ******** folks (particularly the folks who did the XBLA ******** countdown, which is now, uhh, the Fade LLC XBLA countdown) about using the charts in some way or partnering in a business way.
But it's becoming increasingly clear (esp. after a few arguments with them) that they are willing to put out pretty abstractly interpolated numbers, with significantly varying degrees of accuracy, and then retreat to 'all charts are inaccurate - deal with it' rhetoric.
See the iPhone charts, which already got some press:
http://toucharcade.com/2009/09/08/top-iphone-game-sales-numbers-revealed-no-not-really/
TouchArcade themselves commented: "We contacted a number of the developers on the list, and so far, all of them report the numbers are well off target. Ethan Nicholas reports the sales of iShoot [$1.99] to be at 550,000 copies total so far again well off the 843,000 listed above. Illusion Labs (TouchGrind [$4.99]) and MikaMobile (Zombieville [$1.99]) have also responded stating their internal numbers are also well off Fade's. Also absent from the list are Flick Fishing which has sold over 1 million copies and Crash Bandicoot which topped Apple's own list of all-time top app sales."
It was a little bit better when at least the stats were out there and weren't being charged for. But if you're going to pay to get median WiiWare or iPhone numbers which then make you go in the wrong direction with your business, that's pretty galling.
Since VGC are often the only exact numbers out there, they have to be taken seriously as at least something to rebut. And in researching a GDC talk I did this year on digital download numbers (in which I cited carefully and gave pretty broad ranges, which is all you can do at this point), I looked at both XBLA and WiiWare charts from VGC - which are now the Fade LLC charts. The slides are up and you can probably work out what I said from them:
http://www.slideshare.net/simoniker/independent-games-sales-stats-101
(Honestly, I don't think it's easy for NPD to do a good job of most digital charts, either, given lack of reporting publishers and NDAs. So in this case, I'm not even sticking up for NPD over anyone else. If you look around you'll see a lot of people are making noises about digital charts, including Chart Track, but nobody is really getting it together, because there's no compelling commercial reason for platform holders and distribution points to give this info up.)
Yes, it definitely is ioi. Walton, I recommend you check yourself, and that you do it immediately. Your delusional rants and outright lies aren't cute, nor will they be tolerated.
Subscribers to The NPD Group's DDL reports know this research isn't anything more than feedback from its panelists. This is something The NPD Group has been doing for approximately two years, and isn't new news. Any title-level research NPD obtains directly from the publishers and/or mfctrs. would be questionable because it is not from a third party, which is likely the reason why NPD doesn't release anything into the public domain. This would be comparable to issuing news on shipment numbers: practically useless. The best The NPD Group could do would be to keep it internal and use it as part of a larger calibration tool - not that I know any of this for sure, but I'm just saying.
Subscribers to The NPD Group's DDL reports know this research isn't anything more than feedback from its panelists. This is something The NPD Group has been doing for approximately two years, and isn't new news. Any title-level research NPD obtains directly from the publishers and/or mfctrs. would be questionable because it is not from a third party, which is likely the reason why NPD doesn't release anything into the public domain. This would be comparable to issuing news on shipment numbers: practically useless. The best The NPD Group could do would be to keep it internal and use it as part of a larger calibration tool - not that I know any of this for sure, but I'm just saying.
phisheep
NeoGAF's Chief Barrister
Many years ago I worked in a similar business, where there was a monopoly supplier of the 'official' aggregated and sampled sales figures. It wasn't video games and it wasn't the USA - but it was similar enough.
I did manage to lay my hands on the raw data and official algorithms and had dreadful difficulty reproducing the published numbers. Both statistically and algorithmically the official numbers weren't very good at all. Only time in my life I've had to deliberately insert bugs in a program to make it 'work' - horrible. Trouble is, there was a whole set of subindustries built around the wrong-but-official figures, which seemed to me then and still seems now to be fundamentally wrong.
As a result I am all in favour of multiple sources of these things, as in Japan, and not at all in favour of monopoly suppliers.
More on-topic - it seems to me that digital downloads is difficult to do properly. If you merely aggregate supplier figures you aren't really adding anything, even independence; and if you do it by sampling you either change the panel every time and get no consistent measurement of the long tail niche stuff, or you have a relatively static panel that - because you keep reminding them about downloads - consistently downloads more than the population at large and overstates all the figures.
Sampling through ISPs might work, but raises all sorts of tricky privacy issues.
So even if it does fly, I probably won't believe a word of it.
EDIT: missed theguy's post just above, which sounds right to me
I did manage to lay my hands on the raw data and official algorithms and had dreadful difficulty reproducing the published numbers. Both statistically and algorithmically the official numbers weren't very good at all. Only time in my life I've had to deliberately insert bugs in a program to make it 'work' - horrible. Trouble is, there was a whole set of subindustries built around the wrong-but-official figures, which seemed to me then and still seems now to be fundamentally wrong.
As a result I am all in favour of multiple sources of these things, as in Japan, and not at all in favour of monopoly suppliers.
More on-topic - it seems to me that digital downloads is difficult to do properly. If you merely aggregate supplier figures you aren't really adding anything, even independence; and if you do it by sampling you either change the panel every time and get no consistent measurement of the long tail niche stuff, or you have a relatively static panel that - because you keep reminding them about downloads - consistently downloads more than the population at large and overstates all the figures.
Sampling through ISPs might work, but raises all sorts of tricky privacy issues.
So even if it does fly, I probably won't believe a word of it.
EDIT: missed theguy's post just above, which sounds right to me
Mr Killemgood
Member
Hello everyone,
My name is Ben, and I'm one of the analysts that works with the methodology of FADE and the reports that we're working to produce. It seems there have been a few mis-posts by people who work with us, and maybe there's some misunderstanding.
First off, I want to thank everyone for the fantastic conversation about digital download sales. It's obviously something that there's a lot of interest in, but very few people have ever attempted to build out due to multiple reasons. I am of the train of thought of 'If it was economically viable to produce the charts, they'd already exist'. But for one reason or another, sadly, no one has done it.
I started working on XBLA charts almost 2 years ago as a hobby to get away from my current job that was totally unrelated to my field. When I started, I was very surprised that few if anyone had attempted to establish a way to aggregate XBLA sales, outside of Major Nelson (who has actuals). Unfortunately, there are reasons behind this: No distributor is going to come out and openly share numbers because it most likely goes against their contracts with publishers and developers, creating a headache for data aggregation.
Over time, I honed my numbers, got better at them, and continued to publish the charts at VGC. Not because I work or am affiliated with them in any major way, but because it was just there, and I wasn't a frequent poster on GAF.
Likewise, I found other people that had an interest and ability to produce data for other marks, such as WiiWare/VC, and still others like Brett that could contribute website development, which is something I cannot do. Interestingly enough, I've seen other GAFers attempt to aggregate their data using similar methods as what we do. But unfortunately, that market is not as easy to aggregate as XBLA is.
For the argument about what Brett/ioi does with the project:
He has lent his help to develop the website at *******.com. That is the only way he is involved in the project. At all. I work with the XBLA, Android and iPhone data, another person does WiiWare/VC, and a 3rd person helps produce the reports to remove possible bias through our peer review process.
I cannot understate the complexity of how hard this info is to come by, which is why I believe no one has done it. There's no golden goose when it comes to DD data. Rlan has done a fantastic job on posting XBLA data for GameSetWatch/Gamasutra, but at the same time, his lists are only top-10, and producing data for all 240+ titles may be difficult to do. On my end, I developed a methodology using gamercards in a similar way that joaomgc has done here at NeoGAF. In fact, I would like to say his methodology was the reason I even started XBLA aggregation.
I understand our iPhone data has come under scrutiny, and I take all the blame for this. The iPhone is a very difficult market to aggregate because it's much, much larger than any other DD channel at the moment. In August, we tracked over 15,602 titles for the iPhone in which we'll release a report Monday concerning. Because of this, building estimates for a market of such volume is a nightmare. What works for 1 game will not work for another game, and so on. We feel that, going forward, our iPhone work will continue to improve thanks to developer feedback, and continuing work on the market. We've been working with iPhone data for 2-3 months now, and I know that it was 4-5 months before XBLA data began to take a shape that I could publish weekly data and have it match Major Nelson's top-10 list, except with numbers.
Unfortunately, because of the size of the market and the resources required to improve and expand coverage we've decided the best way to make our data better is to charge for it.
Again, I apologize for this Brett/ioi debacle. He doesn't speak for our company. I do. We'll work with these issues, to promote a better understanding of the DD market.
My name is Ben, and I'm one of the analysts that works with the methodology of FADE and the reports that we're working to produce. It seems there have been a few mis-posts by people who work with us, and maybe there's some misunderstanding.
First off, I want to thank everyone for the fantastic conversation about digital download sales. It's obviously something that there's a lot of interest in, but very few people have ever attempted to build out due to multiple reasons. I am of the train of thought of 'If it was economically viable to produce the charts, they'd already exist'. But for one reason or another, sadly, no one has done it.
I started working on XBLA charts almost 2 years ago as a hobby to get away from my current job that was totally unrelated to my field. When I started, I was very surprised that few if anyone had attempted to establish a way to aggregate XBLA sales, outside of Major Nelson (who has actuals). Unfortunately, there are reasons behind this: No distributor is going to come out and openly share numbers because it most likely goes against their contracts with publishers and developers, creating a headache for data aggregation.
Over time, I honed my numbers, got better at them, and continued to publish the charts at VGC. Not because I work or am affiliated with them in any major way, but because it was just there, and I wasn't a frequent poster on GAF.
Likewise, I found other people that had an interest and ability to produce data for other marks, such as WiiWare/VC, and still others like Brett that could contribute website development, which is something I cannot do. Interestingly enough, I've seen other GAFers attempt to aggregate their data using similar methods as what we do. But unfortunately, that market is not as easy to aggregate as XBLA is.
For the argument about what Brett/ioi does with the project:
He has lent his help to develop the website at *******.com. That is the only way he is involved in the project. At all. I work with the XBLA, Android and iPhone data, another person does WiiWare/VC, and a 3rd person helps produce the reports to remove possible bias through our peer review process.
I cannot understate the complexity of how hard this info is to come by, which is why I believe no one has done it. There's no golden goose when it comes to DD data. Rlan has done a fantastic job on posting XBLA data for GameSetWatch/Gamasutra, but at the same time, his lists are only top-10, and producing data for all 240+ titles may be difficult to do. On my end, I developed a methodology using gamercards in a similar way that joaomgc has done here at NeoGAF. In fact, I would like to say his methodology was the reason I even started XBLA aggregation.
I understand our iPhone data has come under scrutiny, and I take all the blame for this. The iPhone is a very difficult market to aggregate because it's much, much larger than any other DD channel at the moment. In August, we tracked over 15,602 titles for the iPhone in which we'll release a report Monday concerning. Because of this, building estimates for a market of such volume is a nightmare. What works for 1 game will not work for another game, and so on. We feel that, going forward, our iPhone work will continue to improve thanks to developer feedback, and continuing work on the market. We've been working with iPhone data for 2-3 months now, and I know that it was 4-5 months before XBLA data began to take a shape that I could publish weekly data and have it match Major Nelson's top-10 list, except with numbers.
Unfortunately, because of the size of the market and the resources required to improve and expand coverage we've decided the best way to make our data better is to charge for it.
Again, I apologize for this Brett/ioi debacle. He doesn't speak for our company. I do. We'll work with these issues, to promote a better understanding of the DD market.
Hcoregamer00
The 'H' stands for hentai.
This thread is fascinating on multiple levels.
Granted, the best part is watching the "misinformation" happen right before me with a suicide account.
Granted, the best part is watching the "misinformation" happen right before me with a suicide account.
Mr Killemgood said:
I'm sorry, are you, Mr. Ben, the owner of "Mr Killemgood" account, or are you "temporarily borrowing the account", just like ioi "borrowed" "alicemudgarden" account ? Just wanted to make sure of it before we go any further discussing things.
If Mr Ben=Mr Killemgood, can you explain this post :
found here
Based on this post, which is less than a month old, you seem to work (present tense, not past tense) for a studio, and I am not sure the relationship between your studio and FADE LLC, since I get the sense that the studio you work now is something like a developer, not Business Intelligence company like FADE LLC (I could be wrong, hence my asking questions)
Just curious, how much you charge for that iphone data ?Mr Killemgood said:
Is it just iphone that was charged (because of complexity), while XBLA, Android, WiiWare/VC remain free ?
IMO, you should focus on getting your algorithm closer/better first, before starting charging for them. I am sure people are more willing to pay for your data IF they are convinced that the QUALITY (accuracy) is there first, not later.
Back on subject, it appears that NPD's "methodology" for DD sales is still going to be based off of a panel of gamers which will be a minute fraction of the total audience. As such it will be basically useless. Until the system manufacturers have a compelling reason to share their internal numbers for DD sales, we're not going to have anything but vague, scatter shot accounts of how DD sales are going.
Mr Killemgood said:
so you will charge for the fake numbers? awesome.
Is fake too strong of an word? +60% difference in real vs your numbers for iphone data is not fake really - it is outright and blatant LIE. You are selling falsified information to your customers who might make business decissions - based on falsified data.
Hopefully you will get sued and your business will shut down. It is one thing to try to make sense of limited data out of curiosity and hobby, and another to claim that data is real and within reasonable %, right.
What some have been saying is what I agree..
Let's say if I go to one of the many sites that provides info tracking...movies' performance, and type...let's say 'Gone with the Wind':
I don't mind them getting paid or charging for what they do, but the fact that ..(again using movies as an example) anyone can get the Top 100 movies at the box-office, for free, every week, at any moment..yet for games the most "we" (i.e. non-payers) can hope to obtain is a Top 10 out of hundreds of games is just ..weird.
Heck, I think Dragona asked earlier why in Japan, 3 different companies track information on a weekly basis and is released to the public eye..yet there's no such thing for the US....what's the difference?
Sorry to sound like a douche, but..
If there's misinformation is because NPD is causing it...the same ones that want to "stop it"..by releasing more ..limited to many..numbers; because even magazines ..which can pay (and even pay) for NPD data, can't post the numbers or any data from them, causing even more misinformation.
Let's say if I go to one of the many sites that provides info tracking...movies' performance, and type...let's say 'Gone with the Wind':
...yet, there's no such things for video games...and the few that is out there, is tracked by NPD..which releases just sales numbers, sometimes ..and even charge for it.
I don't mind them getting paid or charging for what they do, but the fact that ..(again using movies as an example) anyone can get the Top 100 movies at the box-office, for free, every week, at any moment..yet for games the most "we" (i.e. non-payers) can hope to obtain is a Top 10 out of hundreds of games is just ..weird.
Heck, I think Dragona asked earlier why in Japan, 3 different companies track information on a weekly basis and is released to the public eye..yet there's no such thing for the US....what's the difference?
Sorry to sound like a douche, but..
If there's misinformation is because NPD is causing it...the same ones that want to "stop it"..by releasing more ..limited to many..numbers; because even magazines ..which can pay (and even pay) for NPD data, can't post the numbers or any data from them, causing even more misinformation.
outlawauron
Member
Oh wow, this thread has had some interesting discussion.
Anyways, I'd like to see NPDs charts, and I hope they can get the big three to spill total sold rather than just a top 10, etc.
Anyways, I'd like to see NPDs charts, and I hope they can get the big three to spill total sold rather than just a top 10, etc.
phisheep
NeoGAF's Chief Barrister
I think there is a more fundamental problem with doing download sales. The problem is that the publisher always knows the real number sold.
This doesn't happen with retail. Sure, the publisher knows what he has sold to retail, which puts a maximum on the sold-to-customer number - but he doesn't know the real retail figure (and no individual retail chain does either). So retail sampling like NPD has value to him. There is no such value to the publisher in sampled download figures, because he knows the answer.
And because of that, if download figures are sampled and published, the publishers will always call them out if they are low. But the publishers will never call them out if they are high - it wouldn't be in their interests to.
So the published figures will get a lot of bashing for inaccuracy, all in one direction. They'll lose credibility as a result. Because they don't have credibility no one will want to buy them, not for serious money.
I've probably overstated the case to make my point - but the fact that someone out there knows the real figures is bound to affect the commercial viability of sampling of downloads.
(EDIT: just to clarify - of course there are other interests at play, in competitor sales, market analysis and so on, not just publishers for sales of their own games.)
This doesn't happen with retail. Sure, the publisher knows what he has sold to retail, which puts a maximum on the sold-to-customer number - but he doesn't know the real retail figure (and no individual retail chain does either). So retail sampling like NPD has value to him. There is no such value to the publisher in sampled download figures, because he knows the answer.
And because of that, if download figures are sampled and published, the publishers will always call them out if they are low. But the publishers will never call them out if they are high - it wouldn't be in their interests to.
So the published figures will get a lot of bashing for inaccuracy, all in one direction. They'll lose credibility as a result. Because they don't have credibility no one will want to buy them, not for serious money.
I've probably overstated the case to make my point - but the fact that someone out there knows the real figures is bound to affect the commercial viability of sampling of downloads.
(EDIT: just to clarify - of course there are other interests at play, in competitor sales, market analysis and so on, not just publishers for sales of their own games.)
Thunder Monkey
Banned
Sad little ioi.
Can't give it up. Must always try to get the last word. Unfortunately... that can never be.
Can't give it up. Must always try to get the last word. Unfortunately... that can never be.
Alright, I'm at my post now
I've been doing the Top 10 Xbox Live Arcade sales over the past 6 months. Initially it was a fairly simple and straightforward look at the Major Nelson Top 10, but since I've been delving far deeper than I originally planned, and now I've been using the statistics from most XBLA Leaderboard information.
Now I'm quite clear on the possible discrepancies that are there by using this data - there are problems with multiple accounts on the same system being recognized, there are some people you'll see who only have "0" of anything, and if that's you you're not even playing the game. I'm quite open about it, but I feel that in comparison to everything else available it is the best way to do it.
In the past I've been looking at the Leaderboards over a whole month which can cause some problems. There are also games which do not accurately show Leaderboards - Battlefield for example only shows 250,000 players when we know it's in far excess of that, or a game like Puzzle Quest which shows ONLY online multiplayer leaderboards and nothing in regards to single player. What I've been doing for September is getting the Leaderboard stats for EACH week and then putting them against the Major Nelson Top 10s, and from what I've seen they've been quite accurate against Major Nelson's Top 10. So I'm getting better and better in that regard.
I've found the MyGamerCard and other GamerCard places to be largely inaccurate. Take Shadow Complex and Trials HD for example - MyGamerCard and Giant Bomb show 44,804 and 7,275 players for Shadow Complex, and 40,640 and 2,665 for Trials. Even for a percentage of overall sales, to be used to figure out how well the games have done, it's still wildly inaccurate based off the Major Nelson Top 10 and The Leaderboards. '
Trials has been on the top of the Major Nelson list for the last two weeks - and from the leaderboards it's got a whole lot more people who have played it. Going by the other data you'd think Shadow Complex would have done better.
But as Killemgood mentioned, this way I cannot attest to every single digital title. A lot of games have leaderboards which just don't work correctly. There is an overally list of games I could use, but it would be a fraction of the total. It would also include games that have been played via pack ins like Pac Man CE.
PSN is even more difficult - it's fantastic that Sony PR contacts me and reveals the Top 10 PSN / PSOne Classics every month, but almost all games have horrible Leaderboard support, only showing the Top 1,000 players, and that sucks. Who knows how well WiiWare actually does.
But again, my stats are best for an overall look at the Top 10, and a little more around it, but for new releases i think I'm doing a damn good job of working it out.
Now all I need to do is actually get more people interested in reading it all
I've been doing the Top 10 Xbox Live Arcade sales over the past 6 months. Initially it was a fairly simple and straightforward look at the Major Nelson Top 10, but since I've been delving far deeper than I originally planned, and now I've been using the statistics from most XBLA Leaderboard information.
Now I'm quite clear on the possible discrepancies that are there by using this data - there are problems with multiple accounts on the same system being recognized, there are some people you'll see who only have "0" of anything, and if that's you you're not even playing the game. I'm quite open about it, but I feel that in comparison to everything else available it is the best way to do it.
In the past I've been looking at the Leaderboards over a whole month which can cause some problems. There are also games which do not accurately show Leaderboards - Battlefield for example only shows 250,000 players when we know it's in far excess of that, or a game like Puzzle Quest which shows ONLY online multiplayer leaderboards and nothing in regards to single player. What I've been doing for September is getting the Leaderboard stats for EACH week and then putting them against the Major Nelson Top 10s, and from what I've seen they've been quite accurate against Major Nelson's Top 10. So I'm getting better and better in that regard.
I've found the MyGamerCard and other GamerCard places to be largely inaccurate. Take Shadow Complex and Trials HD for example - MyGamerCard and Giant Bomb show 44,804 and 7,275 players for Shadow Complex, and 40,640 and 2,665 for Trials. Even for a percentage of overall sales, to be used to figure out how well the games have done, it's still wildly inaccurate based off the Major Nelson Top 10 and The Leaderboards. '
Trials has been on the top of the Major Nelson list for the last two weeks - and from the leaderboards it's got a whole lot more people who have played it. Going by the other data you'd think Shadow Complex would have done better.
But as Killemgood mentioned, this way I cannot attest to every single digital title. A lot of games have leaderboards which just don't work correctly. There is an overally list of games I could use, but it would be a fraction of the total. It would also include games that have been played via pack ins like Pac Man CE.
PSN is even more difficult - it's fantastic that Sony PR contacts me and reveals the Top 10 PSN / PSOne Classics every month, but almost all games have horrible Leaderboard support, only showing the Top 1,000 players, and that sucks. Who knows how well WiiWare actually does.
But again, my stats are best for an overall look at the Top 10, and a little more around it, but for new releases i think I'm doing a damn good job of working it out.
Now all I need to do is actually get more people interested in reading it all
I was under the impression that the Nintendo/SIRAS company was able to track Nintendo's software sales pretty-much in real-time. There was a thread recently when donny2112 brought it up (and someone he called out supplied the company name and this link).phisheep said:
It was suggested by donny2112 at the time that this is how Nintendo comes up with their own internal numbers (I believe like we've seen when Nintendo announces N sales of Game X in a period that clearly cannot be an NPD tracking month).
phisheep
NeoGAF's Chief Barrister
Oh, you're probably right. I don't keep in touch much with the sales-age stuff - just dropped in because it seemed an interesting topic and I had some related history to spout about.
That's an interesting link. Much more of that gets around and there might be a problem with the commercial viability of independent retail sampling as well.
That's an interesting link. Much more of that gets around and there might be a problem with the commercial viability of independent retail sampling as well.
dialmydrive
Member
The PR link is from 2000. Anything more recent to look at for Nintendo and siras?