I get the point but is not the best UX decision. it's like being on a rollercoaster ride.
I like to see the environment while moving my character. and the game's approach removes this freedom partially.
Well, yes, but as you say it's only partially.
The game makes constant use of "the rule of thirds" in its camera direction, especially when on Agro, so you basically move your character from one side of the screen to the other, when rotating the view.
I also love to constantly rotate the camera and see everything while exploring, so it can take some time to get used to, but I think that (especially coupled with the fact that the horse does
NOT need constant input) it works wonders for this game.
The camera dictates not only the viewpoint, but also the mood and atmosphere, and Shadow of The Colossus is one of the few games (aside from fixed-camera ones) to make effective use of this additional artistic tool.
Lastly (this is not directed to you, it's just a little fixation of mine): when I see on Youtube people going full-speed on the horse by repeatedly tapping X on the controller, I always die a little inside.