....by their fellow knights. Like the one who dismissed Bronn as "just a sellsword" before Bronn mopped the floor with him. Or like Jorah who dismissed the Dothraki because they don't have armor, before one was handily beating him until his arakh got caught not on Jorah's armor, but in his hip bone. (never mind the show where Dothraki mowed down the Lannister army even before the dragon showed up)
Knights are an insular community who look down on everyone outside it.
The same guys who never even consider anyone outside Westeros, at all, despite the fact that an aging Syrio Forel needed just 5 seconds to kill 5 armed and mailed guardsmen while armed only with a stick -- a feat not even remotely matched by anything Dayne, Selmy, or Jaime were ever even RUMORED to have done, much less shown doing.
GRRM is good about making battle "realistic" in the sense that when you get 3+ jobbers together, even the best fighters in the world may well get fucked up (see: The Hound, Brienne). Not Syrio, though, despite not even having a sword.
IMO Syrio is clearly the best swordsman ever depicted in the books (or the show), regardless of how knights exclude anyone not in their gentleman's club.