My PSP & PS1 recommendations:
Alundra (better & more challenging Zelda than most Zeldas)
Arc the Lad 1 (the first game isn't that good. It's passable. But the reason why you should complete it is that you get some great new content unlocked in Arc the Lad II)
Arc the Lad II (the best in the series, a quite dark toned SRPG series)
Arc the Lad III (not quite as good as II, but still great)
Blood Omen: Legacy of Kain (kind of dated gameplay-wise, but it's the start of one of the most epic stories in gaming, so I think you might want to play it)
Breath of Fire III (the best entry in this traditional RPG series)
Castlevania: Symphony of the Night (one of the best 2d action RPGs in existence, still)
Cladun 2X (a rogue-like/dungeon crawler)
Corpse Party (a freaky horror-ish 2d adventure/visual novel game, really good if you think you might enjoy these kinds of games)
Crash Bandicoot 2: Cortex Strikes Back
Crash Bandicoot 3: Warped
Crush (a 3d puzzle platformer with a neat twist)
Darkstalkers Chronicles (a classic fighting game)
Daxter (a decent platformer set in the world of Jak & Daxter)
Disgaea: Afternoon of Darkness
Disgaea Infinite (a fun visual novel, if you enjoy the Disgaea universe)
Dissidia 012 (a really fun 3d combat game with some RPG-lite elements. Don't bother with the first Dissidia, as Dissidia 012 includes the first game's campaigns)
Everybody's Golf series
Everybody's Tennis (a really good tennis game)
EXIT 1 & 2 (decent 2d puzzle platformers)
Fear Effect (these two are interesting experiences, nothing quite like them)
Fear Effect 2: Double Helix
Final Fantasy VII (one of the best in the series)
Final Fantasy VIII (has some stupid plot-twists, but all-in-all it can be an enjoyable FF)
Final Fantasy IX (this has incredibly slow combat, but if you enjoy the world & characters & everything you can do in it, then you might enjoy this a lot)
Final Fantasy Tactics: War of the Lions (one of the best strategy RPGs ever, a very good story)
Front Mission 3 (a great mecha SRPG)
God of War: Chains of Olympus
God of War: Ghost of Sparda (it's a real wonder what Ready at Dawn got out of PSP)
Gods Eater Burst (a more fast-paced Monster Hunter like game)
Grandia (one of those classic JRPGs that everyone should play and not enough people have played)
Gran Turismo PSP (yeah, it's a bit barebones, but it can still be a very enjoyable driving game on the go)
Hoard (a fun dragon-game, really underappreciated & not enough people bought this)
Jak & Daxter: The Lost Frontier (not quite as good as Naughty Dog's original trilogy, but still an enjoyable platformer romp)
Killzone Liberation (even if you dislike or have no interest in PS2/PS3 Killzones, this is very different, a bird's eye view old school shoot em up kind of game)
Klonoa: A Door to Phantomile (classic platformer. A bit on the easy side to get through, but a bit harder if you try to find everything)
Kula World (mind-bendingly awesome puzzle platformer)
Legacy of Kain: Soul Reaver (continues the story of LoK but with a new main character)
Legend of Heroes: Trails in the Sky (a classic-style JRPG for PSP. A bit slow at start and focuses quite a lot on character interactions, but if you like that or can get through that, the end pays off in spades)
Little Big Planet PSP (not as floaty jumping as in the PS3 games, they made LBP work really well on PSP)
LocoRoco 1 & 2 (some of the happiest, most smile-causing platformers ever created. You CANNOT play these games and not become happy and start humming the songs as you are playing the games)
Medievil (a kind of hack n slash/adventure game/platformer hybrid with a goofy main character)
Mega Man: Maverick Hunter X (Mega Man X remake, a really good one of a great game)
Metal Gear Solid: Peacewalker
Monster Hunter Freedom Unite (this series is not for everyone, but if you do get hooked, be prepared to spend hundreds of hours on this)
Motorstorm: Arctic Edge (a really good portable arcade racer)
No Heroes Allowed (I don't know what genre I'd give this, it's a fun game to play on the go nevertheless)
Oddworld: Abe's Oddysee
Oddworld: Abe's Exoddus (these two are classic puzzle platformers, a bit trial & error, but still enjoyable)
Parasite Eve I and/or II (the better first game isn't on European PS Store, sadly, but the second game is pretty good as well)
Patapon 3 (the first two are on there as well, but I think Patapon 3 is so vastly superior that you probably shouldn't bother with the too-grindy prequels)
Persona 3 Portable (imho, the best or perhaps "most playable" version of the game, though YMMV)
Phantom Brave: The Hermuda Triangle (if you liked Disgaea and are pining for more, this offers somewhat similar goofy SRPG goodness)
Prinny: Can I Really Be The Hero?
Prinny 2 (not the bestest games ever, but if you want somewhat challenging action platformers, these are decent alternatives)
Rayman (a challenging platformer)
Resident Evil: Director's Cut (the game that started it all, still holds up pretty well)
Resident Evil 2 (the, imho, better & much improved sequel, one of the best in the whole franchise)
Resident Evil 3: Nemesis
Resistance: Retribution (again, not at all like the console games. A pretty decent 3rd person shooter)
Ridge Racer PSP
Ridge Racer Type 4 (great arcade racers, these two, better than the Vita game)
Riff: Everyday Shooter (a unique somewhat music-based shoot em up)
Rockband Unplugged (as close as you are going to get to the greatness that are Frequency & Amplitude on PS2)
SOCOM: Fireteam Bravo 2 & 3 (surprisingly decent tactical shooters on PSP)
Space Invaders Extreme (Space Invaders brought into current millenia)
Spyro the Dragon Trilogy (another classic PS platformer series)
Syphon Filter Dark Mirror
Syphon Filter Logan's Shadow (pretty decent stealth shooters for PSP from the team that have also made Resistance Retribution and Uncharted Golden Abyss)
Tactics Ogre: Let Us Cling Together (another good SRPG from the makers of Final Fantasy Tactics)
The Warriros PSP (a pretty good beat em up)
Tomba!
Tomba 2! The Evil Swine Return (great platformers that haven't gotten the attention they deserve)
UmJammer Lammy (good rhythm game)
Vagrant Story (a dungeon cralwer action RPG with fucktons of weapon customization & stuff)
Valkyria Chronicles II (the school setting & anime tropes can be a bit annoying and the game is kinda grindy, so if you can stomach those, this can be an enjoyable experience).
What Did I Do to Deserve This My Lord? 1 & 2 (more No More Heroes Allowed)
Wild Arms (classic JRPG)
Wipeout Pulse & Pure (some of the greatest arcade racers ever)
Ys: Oath in Felghana
Ys Seven (these are some of the greatest action RPGs around, really fast-paced, fun & challenging)
I probably missed a few, but I think those can get you started
