Game mode is what you NEED to use when playing video games.
Yes, it will look worse at first... WHY? Because you need to calibrate the TV under Game Mode.
Most default Game Mode settings use horrible settings like 50 sharpness and overly cool color settings.
If you calibrate it right, it will look no different than Movie Mode. There shouldn't be any difference at all when done right, except now, you have less input lag.
If it looks worse, it's because you're not doing it right. Don't blame the mode, don't blame the TV, it's all user at that point.
Just copy the same settings you have for Movie Mode into your Game Mode. You won't be able to adjust certain settings, but that's okay. As long as you do everything else the same, it will end up looking the same. Those things you can't adjust under Game Mode are just processing effects the TV does for certain types of film, as they are all shot with varying degree's of noise and other things. And since games are not made like that, those processes are not needed and only end up making the signal from your controller to the console to the TV take longer.
TL
R - Calibrate yo shit right.