This odd narrative of how MS is scared to add online destruction to the single player portion is a silly excuse. There are technical reasons, but there's no outcry at MS adding destruction to Crackdown SP, just like people aren't upset Drivatars appear in Forza while racing in SP.
It was probably that, combined with the design difficulties of keeping classic agility orbs. But there are a lot of ways around that, and it really shouldn't be taking precedent over the core gameplay, which is the missions (i.e. killing dudes). For another example, perhaps destruction in a building could cause orbs to deactivate, so that you have to wait until a building is repaired to get the orb.
I mean, if Red Faction Guerrilla could do it 8 years ago, with no cloud computing and 4% (yes, 1/24!) of the RAM, sure C3 could at least try. Even if they just make it so SOME of the buildings are destructible, it could add so much to the sandbox. And again, extremely unique. Without that aspect, Crackdown 3 is just another over-the-top open world game. Like Saints Row or Agents of Mayhem, but with worse graphics and less charm. Seems like a waste to have that tech, but only use it for multiplayer.