Question for you luke, matchmaking for Dota. I know that Halo Starcraft 2 and maybe company of hero uses an ELO ranking system it builds a over all ranking of your skill and the goal is to match you where you are in a situation to lose more then you win because it will progressively match you with better and better players until you lose. But for Dota your level can really differ from hero to hero especially in single draft were you might be force to play an unfamiliar hero.
What do you think Valve is doing to rank each player when they don't know what hero you will play in all pick mode. Single draft might give them a way to predict skill. Do you think they are just empirically looking at your "baseball stats" and ranking you that way?
Is that what the level system is going to be?
Not sure how StarCraft handles it, but these systems usually aren't designed to have you lose more than you win, but rather focus players in on an even split between wins and losses.
Halo 2 used an ELO system, Halo 3 relied on Trueskill, but it was a modified version of Trueskill designed to create a more "natural" feeling progression - one of the upshots of Trueskill is that it can rapidly assess how "good" of a player you are and begin matching you there - in Halo 3 we throttled that back, creating a sense of progression.
In Halo: Reach, we used Trueskill as it was intended to be used, a pure matchmaking system where it was focused on finding the best (most competitive, best quality of service) match for a player. Trueskill, presumably like ELO, is trying to dial you in at a .500 win-rate. Players, as it happens, don't like to win half of their games. In fact, I posit that if you ask players what a bad night of any competitive game is, winning half (or losing half of your games) would typically be described as a shitty night of gaming.
I don't have any direct insight into what Valve is doing with DOTA2 matchmaking.
However, were I to guess, Valve has players use self-selection to initially place them in a pool (presumably from that choice at the beginning of the game, where some ELO/Trueskill /(let's just call it ValveSkill since I'm not sure if it's proprietary or not) begins matchmaking you in that Tier.
That self-selected metric is probably obsolesced pretty rapidly by ValveSkill. Then when teams are formed (I'm treating this as the simple, solo queue case), ValveSkill is actually building "teams" it thinks are reasonable and competitive.
Even if ValveSkill was just a scan of Tier (bucketed into 3 groups Low/Med/High) and a cross-section of Wins for those players (You can check player win totals pre-game by bringing up the Scoreboard, and typically you'll see pretty similar Win Totals across both teams) that could result in ok-enough games at this stage.
When people queue as a group, they are clearly manipulating the group up and down through three (player-facing) skill buckets (parenthetical is because they probably have way more subdivisions of skill than 3). E.g., I party with someone in High Tier and they usually pull me up into that game, if I solo queue I match with the other peasants in Medium.
How sticky my ValveSkill number/value is is really interesting. Trueskill is famously sticky, leading to players feeling "stuck" at level X. I'm guessing ValveSkill will have more migration between skill buckets, since DOTA has such a steep learning curve.
How/(if?) they will handle MM values per game mode (CM/SD/AP/???), per player, per Hero played is really interesting. I think because of the point in a game where a Hero is chosen (post matchmaking), there aren't super obvious solutions for how to approximate what a player's actual ValveSkill is per hero and take that into account.
It's not really my place to conjecture how they should/could do that in the thread though. I'm sure if they are interested in it, they have some Top Men over there doing it.