It's invalid, not because of your skin color but because of your analogy. You are comparing systems of oppression to the marginalized meeting with each other. It's ridiculous. And I would also have a few words with Black people who had your same viewpoint.
When I get called nigger, white people can feel bad, feel hurt, feel concern for me, but it's nothing like living it. Hearing someone use that word and you can see the complete hatred in their eyes. That's an awful experience. When I want to have a heart to heart about racism, it's gonna be with other Black people. That ain't segregation.
Of course, there are venues where I do discuss this with white people and other races, but holding a meeting among groups being called segregation or even making the problem worse is laughable.
Every space isn't for everyone and that's not malice. When I was 5ish, My mom told me, I'm going to a baby shower and I couldn't go with her. I asked her why and she said, it's just gonna be women with an expecting mother sharing baby gifts. I understood that, at 5. Is that segregation? I don't think so.
Just to add further clarification, personal experience is unique in that it cannot be read or viewed, only personally experienced. There is a great value to conversations with those that have the same personal firsthand experiences. Others outside that group may feel very strongly, but they can't relate personally on that one axis. How can a person add something when a question in such venues is As a Black woman, how is your life with racism and sexism? Unless you are a Black woman, you can't answer that question. And for others, the next question to ask themselves is, why would you want to take away that personal experience with other Black women (not talking about you here, just using our conversation as an example)?