Not the starting city, but the first you come to I guess. Balmora in Morrowind. So much stuff going on in this city with the imperial fort just outside, the thief guild, the quest guy, the mines, all the rumours about awakening or whatever it was. Simplly makes you incredibly intrigued to continue the game.
There's a very, very comprehensive mod on PC that makes Balmora like twice the size and twice as interesting, really gives it the "lively" feel I've always felt it should have had.
Also, there were wayyyy too many great ones in this thread already listed to repeat them all. It's tough to pick a FAVOURITE.... but I will agree with Rabanastre. I love the name so much I used it for a DnD campaign.
I can only speak for myself, but I'm sure it has something to do with it being a large part of my childhood. FF7, 8, and 9; Legend of Dragoon; Breath of Fire 3; Chrono Cross; all of these amounted for hundreds upon hundreds of hours of my childhood and they always bring fond memories.
Not particularly special, but the music is beyond lovely and the concept is all kinds of awesome. VIII was my first RPG as well, so it gets a few extra points there. Honorable mention to Pallet Town.