The only effect it has is disabling the post-processing that adds the lag in the first place. There isn't any actual degradation of picture quality occurring.
This isn't actually true, at least for some TV's.
You have to remember that the TV has to internalize/process/compute/display a digital signal being sent from your console/PC.
This computation to accurately present an accurate image takes time. Hence where a certain amount of lag comes in. Physics alone prevents 0.0ms input time for example. Even the best, fastest TN panels have some input lag.
So think of it as a linear line: (this is a crude analogy but I think it gets the point across)
Point 0 (no image processed) ------------- Point 1 (accurate image processed) ---------- Point 2 (image that has been processed accurately with additional unneeded post processed stuff)
On crap TV's with crap game mode, the TV's trying to put out the image as fast as possible and thus puts it out some where in between Point 0 and Point 1. Which is why despite any amount of calibration, game mode always looks worse.
Good TV's, Sony Bravia's have a very nice game mode, get pretty damn close to Point 1. To where with calibration, there is no visual difference at all to the human eye.
My Sony Bravia X800D (4k+HDR) does 4k, HDR, and/or Chroma 444 (RGB) at around 30 ms. That's not the greatest, but pretty damn good especially with 444 enhanced HDMI activated.