the Vita's OLED was atrocious. early OLED screens, especially low resolution ones like the Vita's, had way too much brightness variation in the pixels (there is a term for that but I can't remember... it's like clouding but for individual pixels) so if you looked at say a green texture or a green colour in general, it would not look evenly lit or shaded. the colour would look like it has some sort of visual noise
here I threw together a small image of what I mean it would look like. Left the solid red colour, and right how it will look like on an OLED vita. on darker or less saturated colours it would look 10 times worse as well.
on a handheld screen right in front of your face this also again looks worse than on a screen that's a bit farther away from you.
like I said tho, this was not only an issue the Vita had. my old LG G-Flex also had an early OLED of the time and also had this exact issue.
4K output doesn't mean shit. the One S can output 4K but games don't suddenly run at that resolution.