I'm 6'3, so always aisle. The ability to put one of my legs out into the aisle is a teh awesome.
Until the drink cart or another passenger bumps into your leg. But aside from that, yes I completely agree.
Also the exit row should only be for people over 6 feet tall.
Seeing someone a foot shorter than me stretching out in the exit row while I was up to my knees in the window seat was infuriating.