If its a handheld that requires hacking to get emulators on, its no doubt the New 3DS. This is your top option here.
Emulates and runs native pretty much every single Nintendo/Sega/Atari handheld console out there, except for PSP/Vita. Biggest library of emulated and non emulated games you can play on the go.
If you want something that doesn't require hacking, the
GPD XD covers a lot of ground too. And it can support streaming PC / PS4 remote play to the handheld too, so technically you can play almost everything under the sun through that
![Stick out tongue :p :p](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
including PC console emulators. The dual analog sticks are nicer then the slide pad.
On a budget and don't care about hacking, or emulating past PS1 / Snes, and want something very pocket friendly?
Xperia Play. Unless you want to play PSP games, don't go for a PSP when this thing exists. My personal choice for emulation on the go.