I think overall I prefer high challenge games and that sense of accomplishment that comes from doing well in those types of games. My favorite games and series align with that. Monster Hunter, Souls games, SRPGs of old. I like having to learn how the game wants me to play, then doing just that. Obviously when a game poorly instructs you or is poorly built, albeit it with the best of intentions, things get harder when they shouldn't be.
That said, I also have limited time (work, child, life) and can also appreciate lowering difficulty on a game so that I can see it through to the end when I know, in a game by game basis, that that is where my enjoyment will come from. Playing though a game I've long since finished on Normal/Hard when it has been remastered (TLOU for example) on Easy... I have no issues with. Likewise there are times when I have Gamefly'd something and just want to see it through, see all of it, not get hung up in the roadblocks set up by harder difficulties so Easy is what I choose. Genre plays a heavy factor in it too. I am not that interested in competitive games online. I love FIFA and PES but stick to Offline or Co-Op modes. Horror games... scare the shit out of me, but I love the challenge of getting through them, but only in the games that don't force that sense of helplessness on you. I need to feel like I can defend myself rather than constantly run and hide. I can do Dead Space, Evil Within, and loved the Silent Hills games. I am crawling through RE7, but I can't do Alien or Outlast or Amnesia. Even Easy mode wouldn't get me through those.
It really all depends and I think that maybe a lot of us are like this. What I generally don't understand is the frustration some people have for not being able to force a game to play how you want it to play/prefer to play and then calling it "too hard" or "poorly made". Not being able to zerg through something because you have given zero attention to the skill/class system in a game, or are trying to play a stealth game without being stealthy. That mentality, I don't understand.