My smart phone barely has enough battery life to get through a day as it is, when I rarely if every bother to play a game on it.
Phones can compete when their batteries are good enough to keep my phone alive through gaming and also the necessary phone functions that I purchased a smartphone for.
Then, once the battery is good enough to realistically make my phone a worthwhile gaming device, the phone will then have to have original games that are anywhere near the quality of handheld games.
When Shin Megami Tensei 4, Animal Crossing, Etrian Odyssey, 999, TWEWY, Dragon Quest IX, etc. are all built from the ground up for phones, I'll start to take them seriously. But right now, the devs would go bankrupt making games of this quality exclusively for phones, whether they charged $1 or $40.
The closest we have are SMT 1 and 2 ports on phones, and a port of TWEWY that exists only because Square Enix has their heads so far up their own asses that when they eventually suffocate, cause of death is going to be considered a new form of autoerotic asphyxiation.