I might get shit for this, but yes, that can be the case. In one of the games I'm working on, I originally wanted a black main character, but with the art style I want, I found she lost more and more detail the closer I got to her original design. Oftentimes the shadows just made it look awful and in some darker areas the character just looked like a weird moving smudge.
Were I better with lighting, character models and the over all art of the game, I probably could have figured out some compromise that didn't mess up the art style too badly, but sadly I'm just one dude and in the interest of time had to give up on that version of the main character.