My pick from 2004 was Half-Life 2, but MGS3 was a very close 3rd behind that and Metroid Prime 2. It's my favorite Metal Gear game, and certainly was deserving of GOTY, so I have no problem with that choice.
No-brainer. RE4 is my 2005 GOTY and was far and away the best game released that year.
2006- The Legend of Zelda: Twilight Princess
I have plenty of gripes with this game, but it was ultimately my GOTY that year as well. 2006 was a weak year for games overall, it was very transitional, and it's why a really good game that had plenty of flaws was able to win.
Another no-brainer. My game of the generation.
First real disagreement. MGS4 wouldn't have even made my Top 10 that year, I found it to be incredibly disappointing. 2008 didn't have a stand-out game for me (Dead Space was my Game of the Year pick), but I preferred such games as Brawl, GTA IV, Mirror's Edge, and DMC4 to MGS4.
I went Arkham Asylum by the slimmest of margins, but it was pretty much a coin flip as I really liked both games a ton, so I have no qualms at all about Uncharted 2.
Second disagreement, not because I didn't like Mass Effect 2, but because it came out the same year as Super Mario Galaxy 2, Red Dead, Bayonetta, Donkey Kong Country Returns, Deadly Premonition, and a few other games I preferred to it.
My 2011 game of the year by a wide margin over everything else. Once again, completely in agreement.
I don't like this selection. Even though it made my Top 10, and even though it was a really weak year for games, Journey was a nice novelty for me but there were better games. Walking Dead, Mark of the Ninja, ZombiU, Dishonored and Far Cry 3 would have all been more deserving.
My Top 3 games were Zelda: A Link Between Worlds, The Last of Us, and Super Mario 3D World, and I spent hours trying to order them because I loved all three just about equally. I ended up going with Zelda at 1 and Last of Us at 2, but it's still in my Top 3 games of the last generation, so a very well-deserved winner.
Like The Last of Us, Bayonetta 2 was my personal runner-up, so I have no problem at all with it winning. It's a hell of a game.
So, I had the same GOTY as GAF four times, and had the GAF GOTY in my Top 3 four times (3 times as runner-up).
I have three disagreements, and my least-favorite is definitely Metal Gear Solid 4 because I don't really consider it to be a good game in general.