Mushihimesama Futari Black Label
This is just all-around solid. Original mode is the best intro to bullet hells for a newbie - challenging, but not overly complicated. You pick up the scoring mechanic almost immediately and get addicted to vacuuming up the diamonds. Even a klutz like me was able to beat Original mode after ~15 hours of play with no previous bullet hell experience. And then you've got all the hellish challenge of the other modes if you want them.
Espgaluda
Very unique, also easy to get into. I have the PS2 disc but no way to get tolerable graphics out of the PS2. I understand the game now works on MAME, gotta set that up one day if a HD rerelease doesn't make it out first.
Tumiki Fighters
This little free indie is by far the best shmup I've played on PC. OK, that doesn't say much, but I'd even pick TF over many Cave games. Doesn't play like anything else, crazy fun to figure out. I recommend not watching any videos before playing. After putting in 10 hours or so, I was still noticing things about the mechanics that totally changed how I play. Still haven't beat the last boss; this isn't a very easy game, nor one I'd push on someone totally new to shmups.
Ketsui
Super badass soundtrack. Everything about this game is quality (although the X360 port looks a bit ugly, lacking scanline emulation and such). It's way too hard for me, but it's also so good, I don't mind that it kicks my ass in level 3 every time.
Deathsmiles
Unique, attractive, colorful. Fresh theme. Multiple player characters which play very differently from each other. I don't like the fact that the scoring is so rigid and centered on memorization.
Honorable mentions
Espgaluda 2
Work of art. They took Espgaluda and multiplied everything. If my brain ever expanded to the point where it can grasp what's going on in this game, I have no doubt it would be my favorite. It's not just the dodging that is the problem, but juggling two counters and 4..? different character modes.
Progear no Arashi
The graphics and the mood are fantastic, and the game controls well as Cave stuff always does. Contrived, annoying scoring though.
Twinkle Star Sprites
... would it be too much to ask to get a spiritual sequel of this on current systems?
Super Stardust HD
Great sense of weight and exploding lots and lots of stuff. I haven't played it enough to know if it has legs scoring-wise, but feels good man.
Grid Wars 2
Looks like a Geometry Wars clone, but is actually a black hole farming simulator disguised as Geometry Wars. You need a gamepad to play, as you get better you soon realize other controls do not cut it. Also, there's no point playing this just for survival; you have to play for score. (you have to DL this from a 3rd party source since the creator had to pull it after getting IP complaints)