I always thought that different kingdoms in The Witcher series represented different aspects of Poland and its neighbors, same as Elves and Dwarves are there to reflect various facets of discrimination that the Jews (and sometimes the Romani) faced.
Northern kings were sometimes wise, sometimes deeply stupid and short-sighted, sometimes good politicians while also being horrible human beings (and vice versa) - just like Polish rulers of yesteryear. Poland's history is very complicated and rarely straightforwardly black and white. Sapkowski wrote a series as a critique of xenophobic post-communist nation, but never denied its tragic background. This is why it resonates so deeply with the inhabitant of the region - it's not afraid to point-out flaws while never losing the sympathy for the peoples of this world.
People of Color weren't Polish problem, not when the series was written and nor now. As harsh as it may sound for Americans, there're simply too few of them to matter in the national scale. We have much more pressing ethnic issues. What with the Jews and their still-present, unresolved issues, residual after years of co-existence, than holocaust, than communism? What about Romani, marginalized and looked down upon in the same way as it was years ago? Not to mention the Germans, the Russians, The Ukrainians and Lithuanians? The class issues? Sapkowski wrote about all - or most - of that and it will be lost on those who don't know Polish history. The universal themes are easy to grasp, especially if the series borrows quite generously from Western and Northern mythologies too - but those little, region-specific nods are more elusive.
So some Americans (and Western-Europeans) will look for racist themes they understand, like colonialism and slavery, and try to mold the narrative to their experiences. They see all white cast and wonder - where are all the Blacks and Asians? They won't comprehend that these groups of POC are mostly irrelevant to the story Sapkowski and CD Projekt Red wanted to tell. Could they be included? Sure, why not. But their stories wouldn't be explored as the series are not about them this time. Not all racism is about Black people. There was no systematic oppression of them, but it occurred for other ethnicities, those who would be classified as "white" by American society.
Let's imagine what a POC's experience would be in The Witcher, the series that explores racism and it roots (in Eastern European flavor). What a hypothetical Zerricanian merchant would say about his life in Novigrad? Not much probably. He would complain how strange people dress, what silly customs they have and how annoyed he is by those looks he gets every time he walks the streets. Some hostility, but mostly curiosity and ignorance, as the Northerners have no ill history with Zerrikania, no old resentments. Even if some jerks and racists would pick on him, it would be only because they generally hate everything that is different. It wouldn't have complicated backstory that Humans and Non-humans share in this universe. Such portrayal would reflect on what people with different skin-color are perceived in Poland, modern or medieval.
It would be perfectly fine to have a Zerrikanian merchant like this in the game, or a warrior-woman, or even the traveler or scholar. But i disagree with the opinion that omitting this somehow made the game inherently racist. One token character would do nothing to make the game as a whole less white.