I mean, it's really not the same thing. If you're in the majority, then experiencing the culture of the minority is new and exciting. If you're in the minority, you've grown up submerged in the majority's culture. You already know all about eggs and chocolate rabbits and lip service to a reborn Messiah.
Furthermore, if someone in a minority invites you to their celebration, they probably know you're an outsider and want to include you anyway. When someone in the majority does so, it's because they assume you're celebrating the same thing, because 'everyone' celebrates that thing (except those weirdos who don't).
I'm not saying it's justified to go apeshit on a random "Merry Christmas" by any means, but it is a microaggression worth being mindful of. Especially to individuals in particularly fervent areas (eg the Bible Belt).