The Xbox failed in Japan because it wasn't Japanese and you already had One new juggernaut that has proven itself the generation before (Sony) and juggernaut that was advertising a massive comeback that was an established name (Nintendo) and a niche local brand that never really gained any real ground but made it hard for anyone else to come in (Sega) with the other thought to be household name leaving the industry the previous gen (NEC.)
People like to compare Xbox to the Iphone to dismiss the argument that it's because the Xbox wasn't Japanese, but that';s not a logical argument to make, the Iphone was years later, most smartphone companies adapted from its design, and it was in a broader market in an industry considered a necessity more and more not a luxury.
the video game market in Japan was different in America, where there was never really any loyalties even for any one generation.
In America, Atari/Coleco/Intelvision were kings with Coleco being the last one standing. When Nintendo entered people still went back and brought Ataris before the NES finally completely overshadowed it after 88. The the genesis was the main favorite before Sega started making mistakes giving a slight victory to Nintendo at the end, then Sega collapsed and the N64 only made relatively marginal ground as new comer Sony mopped up.
In this type of environment someone like Microsoft coming in with new styles of games, PC developers, and bad ass hardware has a place to jump in, and did.
In Japan you had Nintendo domination for 2 eras with the TG16 by EC possibly seen as the new champion who took too long to come out with a successor and instead focused on ad-ons, then released a flop consoles that crashed and burned. Sega never really managed to get anywhere, leaving a struggling Sega, a new comer, Sony, which was extremely popular with 5x the library, and a Nintendo that entered in late with multiple delays.
Now with Nintendo making a comeback, Sony expected to continue dominating, and the Dreamcast being a non-factor, the Xbox would need to convince the Japanese that the console that has completely different types of games not from japan and only 14% of its games being Japanese was better than the others.
The original Xbox couldn't even beat the 3DO in sales in japan, and 3DO knew the had to get Sanyo on board and to not sell the Korean LG branded 3DO or any other branded 3DO(at the time) in japan because they knew that the 3DO had to be as Japanese as possible to the average Japanese citizen. To the point where they had to pretend half their hits from Crash N Burn to Gex were anime games. But that only worked for a limited time, and once the initial Japanese wave of games started to come in the PSX already had like 4x the library of the 3DO) in japan because 3DO/Panasonic/LG/Sanyo combined couldn't outbid Sony on all those consoles exclusives.
The 360 was and will be the best any American machine will do for the foreseeable future, at least in terms of traditional hardware (unless they have some new massive plan foe the next Xbox which I doubt.)
With the original Xbox barely selling below 500k (3DO close to 800k) Sega dying, the PS2 doubling the PSX, and the GameCube hanging on by a good amount, and the costs of entry into the industry, as well as stocking discs on retails shelves rising insanely, that was the Xbox;s ONLY chance to carve out a foot hold, instead the initial games rush dried up, and everyone else In japan moved so far ahead in retailer presence and software support it was a done brand.
The 360 only beat that number because it not only released first and had tons of Price cuts, but they manged to pay off enough media to get some early pretense and Kinect kind of, sort of, helped later on.
Also in hindsight, the Xbox original losing to the 3DO is really telling since the Xbox had like 8x the attach rate, and software support yet sold nearly half as much during a time the market was bigger.