-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
NPD Prediction Threads: Discussion/Voting for new points systems
- Thread starter donny2112
- Start date
Zynx said:
That's the normal way the monthly predictions are done. That's not stopping. It doesn't work for multiple months, though, as some months (e.g. December) would way skew the results because of how much the top 10 would be off by in absolute terms.
Zynx said:
That is a common fallacy. No one knows what the correct value is beforehand, so using a "Price is Right" method is doomed to failure. Also if certain users started doing that, we could just stop using taking their predictions. Essentially "ban" them from the prediction threads.
Zynx said:
System #2 is the one that would do that. It ranks purely based on how well the user did compared to the "truth."
Zynx said:
It would also have to extract the username that predicted, and be flexible enough to handle a fully spelled out prediction (e.g. 430,225 instead of 430K). Entries that don't follow the format would still need to be entered by hand, but saving some time is better than saving none.
FYI, it probably takes me about 30-45 seconds or less on average to enter in most users' entries, so an automatic method would have to be faster than that on a per entry basis, including the time taken to copy-and-paste into the parser. Pretty much every concept I've come up with for a parser would potentially take about as long to process with all the overhead taken into account.
Cheesemeister said:That'd be a cinch to program.
Would you care to give it a shot?
Earthstrike said:
That's what the "*3" rule under System #2 is. Potentially, we could use the #1-#10 bonus values from System #1 there.
Fredescu said:
ComputerNerd said:
sonycowboy said:
border said:
Pssst. You're posting on a video game message board on the INTERNETS.
Since most prefer system 2, I also vote for system 2.
However, I think that :
- point #1 should be changed from 25 point to lower point (5-10 points). The 25 points is too big a bonus (10%+). This will keep things from competitive until Dec, for the yearly points.
- I agree that there is a bonus for position #1 to #10. Just use the bonus rules from system 1.
- The points calculation should be rounded up to the nearest 2 digit decimal for each component. So, 18.33333333 became 18.33. The way I see it, most of the vote will be very close (for those who did not get the "right order of the h/w position" bonus), and it is important to keep track of the 2 digit after decimal value).
- We need to decide the closing time of this point system discussion. If one week after OP (June 10) sufficiently long ?
donny2112,
After we decided on the actual system and bonus calculation, can you please calculate the points for previous months (each months a separate list) for all predictor ?
However, I think that :
- point #1 should be changed from 25 point to lower point (5-10 points). The 25 points is too big a bonus (10%+). This will keep things from competitive until Dec, for the yearly points.
- I agree that there is a bonus for position #1 to #10. Just use the bonus rules from system 1.
- The points calculation should be rounded up to the nearest 2 digit decimal for each component. So, 18.33333333 became 18.33. The way I see it, most of the vote will be very close (for those who did not get the "right order of the h/w position" bonus), and it is important to keep track of the 2 digit after decimal value).
- We need to decide the closing time of this point system discussion. If one week after OP (June 10) sufficiently long ?
donny2112,
After we decided on the actual system and bonus calculation, can you please calculate the points for previous months (each months a separate list) for all predictor ?
Cheesemeister
Member
donny2112 said:Would you care to give it a shot?
Yeah, sure thing. The actual formula used to do the rankings would be just that -- a formula -- so it'd be easy to change. The initial implementation, of course, will just be the absolute diffs... I think Nov. and Dec. should be worth more in terms of cumulative rankings, as more sales are at stake during those months.
donny2112 said:
Parser flexibility isn't a problem; errors can be reported. Case-insensitivity, subtle variations on system names, interpretation of "K", and order of system predictions are no issue.
ComputerNerd said:
Uh... I'm afraid I've let my website-creating skills lapse. I was thinking about doing such a thing for downloadable game sales tracking, but have been putting it off due to all the homework involved.
apujanata said:
I think 15 points may be more reasonable. That is just over 10%. The most that have gotten the order correct this year is 12/160 in April, so it's not like everyone gets it, anyways. In sonycowboy's original system, there were a lot more possible points due to software, so 25 points in that system wasn't relatively as big a bonus.
apujanata said:
Full precision is kept internally. I would only plan on posting to 2 decimal points, though.
apujanata said:
Based on how the vote is going so far, it may be enough to decide tomorrow.
While there were some votes for the way rankings are already done (and will keep being done) and a few other misc. ideas, the main vote at this point is ...
1 - i
2 - xv
Edit:
Cheesemeister said:
I use Excel for recording the predictions and then move that to Access to tally the rankings. If the parser could just get it into an easy to read format for Excel, that'd be a great first step.
Cheesemeister
Member
donny2112 said:I use Excel for recording the predictions and then move that to Access to tally the rankings. If the parser could just get it into an easy to read format for Excel, that'd be a great first step.
I can do you one better and output the results of applying any formula in nicely-formatted BBCode text.
Cheesemeister said:
The way I currently do System #2 in Access:
1. Read the monthly hardware results from a table and come up with a "system, rank" virtual table for that month.
2. Take all the predictions for that month and come up with a "username, system, rank" table for that month.
3. Compare the "system, rank" values from the virtual table in #1 on a username basis to the table in #2.
4. If they all equal, user gets 25 points. Otherwise, they get 0.
That's the first set of points. The second set is a much easier and straight-forward comparison of the % difference per system on a username basis and multiply that by 20. I then have a query go and sum up the points, grouped by username.
The second part is a straight-forward formula. Do you think it would be worth the trouble for you to program the first part into a parser?
Edit:
OatmealMu said:
:lol bishoptl said something similar about sonycowboy's point system back-in-the-day, too.
Cheesemeister
Member
donny2112 said:
Yeah, that's all simple table sorting and comparison stuff. Very straightforward.
Cheesemeister said:
Cool.
Thanks!Edit:
What about entries that aren't automatically parsed? You mentioned that it would output errors, if it couldn't parse the data, so I can just use my old, decrepit method on those.
Edit2:
Could the parser still have an option to output the username and predictions data? I keep a monthly Excel sheet with most of the prediction results from the last year, and I'd like to keep that current, if possible.
Cheesemeister
Member
donny2112 said:Cool.
Edit:
What about entries that aren't automatically parsed? You mentioned that it would output errors, if it couldn't parse the data, so I can just use my old, decrepit method on those.
Edit2:
Could the parser still have an option to output the username and predictions data? I keep a monthly Excel sheet with most of the prediction results from the last year, and I'd like to keep that current, if possible.
I'll provide a template for users to copy/paste, but will try and make it as robust as possible. Data parsed in can be spit out again with a simple for loop. In what format?
JoshuaJSlone
Member
Hey, as long as we're here arguing pointless stuff anyway, we might as well have a record of whether or not person Z is ever even close to right.border said:
ComputerNerd
Banned
Cheesemeister said:
I can do the website. Keep in mind that it would be extremely simple.
I really like the points idea. System #1
Is it also possible that you could set up a google spreadsheet so we could see all the results?
Can you also post the statistical median to see how the data skews?
How about predicting the top 3 or 5 SW titles? Maybe not numbers but order for additional points?
Is it also possible that you could set up a google spreadsheet so we could see all the results?
Can you also post the statistical median to see how the data skews?
How about predicting the top 3 or 5 SW titles? Maybe not numbers but order for additional points?
sonycowboy
Member
tanod said:
I'm working on this in a larger sense. Let's hold off on this for a little bit. (Also, when I did have the SW as part of the contest, the SW predictions were about a 1/10th of the HW predictions in terms of participation).
sonycowboy said:
That makes sense but with a points system, it could be an easy way to earn an extra 5 points or so. Suppose I predict:
1. Pokemon Diamond
2. Pokemon Pearl
3. Mario Party 8
4. God of War 2
5. Odin Sphere
and the actual results are
1. Odin Sphere
(no point)2. Pokemon Pearl (+1 point)
3. Pokemon Diamond (no point)
4. God of War 2 (+1 point)
5. Mario Party 8 (no point)
so I get 2 bonus points for April.
With bonus points, it gives people an incentive to add their picks but doesn't try to burden anybody with pulling more numbers out of their ass or somebody having to calculate how close the predictors were to the actual sales number and assigning rank.
DayShallCome
Member
I prefer system 2, though I don't like scoring point 1 (bonus for correct order), as I don't really see what it adds as to the system in regards to guaging quality of predictions. Doesn't % tell us all we need to know? If someone doesn't pick the systems in the correct order, than his % difference will account for this, right?
DayShallCome said:
Not with close systems. For April, someone could have predicted 1-5 correct, but put PS3 at 85K and GBA at 83K. Percentage-wise those aren't far off, but it prevented him from getting the order correct. If it helps, you can think of it like bonus points, though it actually does take a good bit of predicting to get the order correct and have decent percentages.
I think it's pretty clear that most are in favor of System #2. With that in mind, here's the more finalized version of System #2.
Rules
1. 15 points for getting hardware positions correct.
2. 20 points * (percentage of correct units for hardware system); anything over 100% difference = 0 points. Maximum points: 140
3. Bonus points for finishing in the top 10 by units for the month:
#1 - 17 points
#2 - 12 points
#3 - 11 points
#4 - 9 points
#5 - 7 points
#6 - 5 points
#7 - 4 points
#8 - 3 points
#9 - 2 points
#10 - 1 point
Using that system, here are the 2007 top ten and the top ten for just the last three months.
NPD Prediction Points Leaders (2007)
1. BuzzJive - 488.71
2. sonycowboy - 487.3
3. argon - 483.09
4. GhaleonEB - 477.65
5. donny2112 - 476.37
6. JoshuaJSlone - 463.1
7. Clever Pun - 462.58
8. starship - 449.25
9. duderon - 448.77
10. Orgen - 448.24
NPD Prediction Points Leaders (3 months)
1. BuzzJive - 368.48
2. donny2112 - 360.13
3. argon - 356.33
4. Jokeropia - 353.8
5. DayShallCome - 352.52
6. Clever Pun - 351.55
7. Heidir - 350.91
8. sonycowboy - 343.28
9. GhaleonEB - 342.8
10. starship - 341.71
Ordered, but unnumbered, points listings for each month can be found here. I plan on updating that page with each new month's data after it's calculated.
DayShallCome
Member
donny2112 said:
Yah, makes sense. But in the case you mentioned, that makes 2k worth 15 points (+ the % difference penalty). I'm not a big fan of doing the ordering, but it isn't a big deal at all. As long as everyone plays by the same rules, it makes no real difference.
Thanks for all the work Donny.
Cheesemeister
Member
Post formatting guidelines for the parser...
List numeric values following system tags on their own lines in a post. All non-numeric characters following the tag will be stripped out, save for the first instance of the letter K in either upper or lower-case, which will be replaced with 3 zeroes. Make predictions by the set deadline, at which point the parser will be run. If a given poster has more than one post with predictions, guesses in subsequent posts will override previous guesses. By the same token, subsequent guesses on the same system within a given post will overwrite previous guesses.
Some examples of valid input follow, but do at least keep them readable for your fellow forum-goers.
[PS2] 123,456
[PSP] 123456
[PS3] 123K
[360] 123k - because of Halo 3 anticipation
[GBA]12 3 Totakeke
[NGC] 1 2 3 4 5 6
[NDS] 123456 (You will say WOW.)
[WII] 123'456
Invalid input:
Sony PlayStation 2: 123,455 plus one
PS Three - 123KK
Weeeeee ~ one hundred twenty-three thousand kajillion units!!!1lol
...
Any lines without valid input, i.e. your comments, will be ignored. Questions, comments?
List numeric values following system tags on their own lines in a post. All non-numeric characters following the tag will be stripped out, save for the first instance of the letter K in either upper or lower-case, which will be replaced with 3 zeroes. Make predictions by the set deadline, at which point the parser will be run. If a given poster has more than one post with predictions, guesses in subsequent posts will override previous guesses. By the same token, subsequent guesses on the same system within a given post will overwrite previous guesses.
Some examples of valid input follow, but do at least keep them readable for your fellow forum-goers.
[PS2] 123,456
[PSP] 123456
[PS3] 123K
[360] 123k - because of Halo 3 anticipation
[GBA]12 3 Totakeke
[NGC] 1 2 3 4 5 6
[NDS] 123456 (You will say WOW.)
[WII] 123'456
Invalid input:
Sony PlayStation 2: 123,455 plus one
PS Three - 123KK
Weeeeee ~ one hundred twenty-three thousand kajillion units!!!1lol
...
Any lines without valid input, i.e. your comments, will be ignored. Questions, comments?
AzureNightmare
Banned
hmm
What is with all of this complicated bull shit? Just take the 6 systems that are relavent..... 360, PS3, Wii, PS2, DS, and PSP. Guess the numbers, and then establish who was closest. Nobody cares how GBA is selling anymore, or GC, or XBOX. It is just annoying to have to think about it. Go with the 6 systems that are still very active and that we can follow. Yes GBA is still around, but nobody gives a ****, and there is no real basis to predict its sales on. Let it die.
AzureNightmare
Banned
dont ask meC4Lukins said:
apujanata said:
I'm pretty sure I used your exact previous suggestion for the bonus points.
Just so it's not too different from what was previously used, I'll just change the #2 to 13 points, so it makes more sense. Cheesemeister said:
If I do the prediction thread, I'll suggest they use that formatting, but I'll still allow them to use their own formatting. It just won't be understood by the parser. Those predictions would then have to be entered by hand (i.e. how previous months have been). The idea is that giving more freedom in how you post predictions gets more participation.
I would guess that most will use the parser format, though, so that'll still be a huge time-saver. Thanks!
C4Lukins said:
If I do the prediction thread, I'll still use GBA all of this year. If you don't predict a GBA number, I'll just assume you thought it would sell zero that month.
Bildi said:Nice work donny.
It's mostly based off of sonycowboy's previous points system with input from others, too, but thanks!