I don't know if "hardest" seems apt. It's only been the last few years that the N64 even had open source efforts being worked on, and for that direction of emulation, it's been the one to net the greatest results. Dolphin, PPSSPP, and PCSX2 have a lot of credit having to go to the fact they had constant builds made daily, and this was due to the fact they were open source. I think that direction will greatly help Xenia, RPCS3, and Citra obtain adequate emulation of Xbox 360, PS3, and 3DS respectfully.
Even there though I worry that Xbox 360 emulation will sort of be ignored by the community, just like Xbox emulation was. It seems in emulation circles it's mostly about Nintendo and Sony hardware.