Why would anyone take that literally? It's rap music, everyone embellishes.
It's a silly concept. People actually respect rappers more if they've been involved in gangs.. lol.
At the very least though, many rappers are from grimy hoods. It's known Drake is a suburban Canadian who was on TV as a teenager.. so he get's made fun of for it. Same with his "Started from the Bottom" shtick of course.