Most casual clothes really doesn't need ironing. If you hang to dry or pull it out of the dryer right away, at most it'll be slightly rumpled, in a way that will smooth out pretty quickly as you wear it. I'll iron or steam chinos or dress pants/shirts, but generally not oxford shirts – they look fine not perfectly crisp. I don't know what kind of denim people are wearing that has any real wrinkle problem. Probably heavier weights are better about this?
As for the "never wash denim" thing, I've gone to the extreme in the past, but softened on that. Washing after every wear is way too much, but if they smell at all or look dirty, I just wash (and air-dry) them. That still only ends up being once every month or two, unless I've gotten them particularly dirty or sweaty for some reason. They don't smell laundry-fresh or anything, but they also don't have any BO odor. Wearing and changing my underwear daily seems to be enough for me to avoid that. Your mileage may vary based on your body chemistry and habits.
Washing jeans too much probably wears them out prematurely, but leaving them grimy with dirt and body oils embedded in the fabric wears them out too. I'm OK not having the sickest fades.
Every shirt I own cost me more than 60 quid (🏀 not way I'm letting myself be seen in them where they are anything less than crisp.
I don't iron everything, if I'm wearing a sweatshirt there's not a chance I'm ironing the t shirt underneath. And I have a couple of sweatshirts that don't need ironing. It's a garment by garment thing imo and if you're actually wearing it out in public. If you're dossing about at home who gives a fuck.