This is exactly the point I'm trying to make. A lot of people use the word almost exclusively for black people and they don't even realize that they do it. This is an example of casual racism, because you can't really call someone out on it because then they become defensive and use the argument that "thug" isn't a racist term. In which they are absolutely correct, but it doesn't change the fact that that's the way they were using it. I'm not talking about you specifically, and if you used it for people of all races than that's a different matter.
I know you were making that point. Thug isn't racialized, but the individual sees more blacks fitting the bill to be called a thug than whites. Thus the frequency of blacks being called thugs indicates casual racism as expressed through the word thug.
But do you see my point? Because it can be coded language, there is the pitfall of people trying to decypher it and paint the user of that language as casually to ostentatiously racist. The best you can do is point out that a word has become coded and hope the person is honest. If they really are racist, or have some sort of ignorance or bias they may need to confront. It will reveal itself more apparently eventually. But seeing people marked prematurely and stonewalled over a perceived leveraging of coded language is frustrating. And that extends beyond just racial examples. It extends into an array of topics.