I still have to look down at my 360 pad to realize the button layout. Though I imagine a large part of that confusion stems from barely touching it since I use it for PC.
I have the hardest time with quick time events on an 360 controller for some reason. I fail them every time because I have to look down at it. The SNES controller completely ruined my knowledge of where A, B, X, and Y are.