I kinda feel like you didn't play Xillia 1 & 2 if you consider them embarrassing garbage compared to Graces F.
Both are equally good/bad depending on your tolerance for them. I mean, at least I didn't have to stomach 40 hours of "FRIENDSHIP WILL SAVE US" through Xillia
I feel like this series gathers internet rhetoric when people talk about them and it just gets endlessly repeated until people believe it to be true. It gets me most since I recently played through Graces -> Berseria over the past 6 months or so and there's no real quality fluctuations until Zestiria, which took a massive nosedive in story terms. Sure, Graces had some awful production values and Xillia's story felt a bit rushed, but there was nothing noticeable - certainly no "this one was okay and this one was embarrassing garbage"-esque changes in quality.