Pence is a man stuck in between two worlds, one where he fully expected Donald Trump to lose, elevating him to the defacto Republican nominee in 2020, and another where Trump bizarrely lucks into the White House. Nobody respects Pence because Pence doesn't really exist. He's not even there. When they see him walking around the White House, I'm sure they're like "it's that guy again. Who is he?"
He's as much a non-factor in the White House as Sean Spicer is. In fact, I fully suspect Kellyanne Conway has a lot more influence.