i think nintendo made the mistake of trying to make an iphone out of the 3ds. the ds did well despite not having the same features the psp had, and it did well despite being weird, outdated technology. it just had the games people wanted, and they were affordable.
the things in common all nintendo handhelds had up until the 3ds was: portability (general size), price (affordable software and hardware), accessibility (in terms of controls and games), and battery life.
the 3ds is a $170-$200 machine that was $250 at launch. it had weird screen issues and loose hinges. the battery life was short, and didn't make it a good portable. the games for the machine came from other systems until christmas, and even after that, there was nothing on the machine but a bunch of console-lite titles.
to top it off, the games are not affordable, even compared to their previous software.
the price needs to come down in general. a $99.99 3ds with $20-$30 would be a much bigger hit in the united states, but the system also needs its reason to live like tetris and pokemon did for the game boy, or nintendogs and brain age did for the ds.