One thing I think is better is that the fast forwarding is much faster. If you ever listened to podcasts or radio shows on your Go, you know how shitty it was to fast forward to 2 or 3 hours in. It was slow as hell. Something I personally hated about the Go was also the fact that accidentally hitting the shoulder button would always change the track, so if I had just fast forwarded a long podcast or radio show to the right spot, then hit the shoulder button, I'd have to start all over again. So for me that's a plus but others have stated they don't like that it doesn't change tracks that way.
Other than that I can't think of any disadvantages that the Vita has. It still saves your exact spot even when you close the music app, so that's still in tact.