If you're only getting Azurite, then you're not in the right areas yet. But, you will be soon. You'll be swimming in master and flawless raw materials, and there will be a ton of high end/top end weaponry for you to break down.
I gotta admit that the I'm the RNG's bitch right now. Holy hell I can't find anything good and I've been using the same crap for many levels now. I don't know what people mean by making better gear through Blacksmithing. The best blade I've found has been made of Azurite and that won't make anything remotely better than the drops I've found already. I wonder if there's just a lot of hyperbole going on, because my experience is so completely different than what I'm reading here. I've died tons of times to damn everything, my gear is crappy, blacksmithing isn't helping any since I can't get any decent parts to make anything good. So frustrating. I'm level 21 (60 hours in) and I had to buy a blue sword from the store because nothing I've found anywhere was even remotely as good. <sniff, sniff>
Even without perfect materials it's still very possible to break the game, though. Craft a couple sets that have + to hp/mana regen, and bolster up your phys/crit/magic damage %, and you're set.
Definitely not hyperbole. I crafted a full set of armor (still using purple drops I've found for weapons - which I have found a ton of, all very good weapons at that) and it takes 3 mages pounding me with whirlwinds/fireballs to get me to use a potion as I regen through everything else.
I'm level 39 at the moment and I'm just tearing stuff up as finesse/might. I think that once I hit 40, though, I'm going to respec to universalist (37 points in might/finesse/sorcery) for kicks.