If someone is seriously injures or even dies, I'm betting he tried to stop it but couldn't.
That's something that could leave him haunted, that even with all the power of the Hulk, he couldn't stop someone he cares(?) about from being hurt/killed.
I'm guessing it's from having SW messing with his head. The shot of Hulk's hand out in front of Widow seems like a scene where Hulk might be getting confused/losing control of who he is and recognizes. Cue the Hulk/IM battle in the streets. After, Banner is all shaken up after having been fucked with and put out of control like that.