Graphics Horse said:
Yep, the other two axes are easy as the accellerometers will detect gravitational pull, but the x,z plane can't be done that way.
I'm not totally sure I get your example, so imagine holding your arm out and holding the stick in any way you choose, then walking around the stick so it rotates but doesn't move.
Warning, tl;dr post incoming.
Here's my take on it (not necessarily your example, but what looks to be possible and how it could work).
It's clear that the ball helps track the movements and position in the x,y plane. It's a no-brainer and been clearly shown in the patents anyway. Now, could depth be tracked (at least partially) by the camera as well? The perceived size vs it's "real size" (which could be coded in the software) might help track the position in the z plane. If the ball has a diameter of 5cm IRL, maybe the camera can measure its perceived size. Now I'm not sure the resolution of the camera would be enough to track subtle movements, but it might help to give some general reference points and correct errors.
Some people mentionned that the glowing ball might use an array of leds. Let's say that the PSeye sees it as a perfect circle (again, with a known size) if you hold the wand vertically. If you start pointing at the screen (leaning forward), it will look like an ellipse. This could also help in determining the relative position of the wand. Same stuff in case of your example. But yeah this is just speculation because we don't know if there really is an array of leds in there, and its shape.
Simple rotations or angles can be tracked by the gyros. If the ball's position does not change at all, the camera won't help. However, it's a really simple movement with not many variables and since the position (in 3D space) of the controller does not change, the gyro's precision might be more than enough.
Now if you do a more complex movement in 3D space, the gyros might not be accurate enough and errors will accumulate over time (which is WMP's biggest problem if you don't use the pointer function). The big advantage here is that the light ball permanently helps as a reference point as long as it is not hidden to the PSeye.
Let's say you hold the wand vertically, and move it to point directly towards the screen so its position in the x plane doesn't change one bit. If you combine the data from the gyro (which might be a little inaccurate) with the change in the y plane, you have both start and end points with the camera, plus the rotation/speed from the gyros. If the camera cannot track the z position at all, the combination of both the gyros and the camera should work. Because you have a start point, an end point, and the movement which has been used to get from one to another. So I imagine the exact z position can be deduced from both sets of data. It all relies on calculations however, and not an absolute detection of the z position, which can not be determined either by the camera or the gyros.
That's where the ultrasonic stuff kicks in (if it's used. They didnt mention it at all in the demo, but since it was described in the patent filing I doubt they took it out). Used alone, it should be enough to determine the z position. That said, I'm not sure a small change could be picked up by the microphones. But again, if you combine this with the stuff I've mentionned above, it could allow a great precision.
Now there's a last point with that ultrasonic stuff. Since the PSeye has an array of mics, it might even help to track more than just the z position. Much like the pointing function of the wiimote. The nature of the triangle formed by the 2 leds and the wiimote camera can be used to determine the position in all three planes.
Now the spacing between the mics is rather small, and the use of sound should be less precise and more prone to artifcats than the IR leds/camera used for the Wii. So I'm not sure it does work like this for the Sonywand.
All in all, this controller uses a large variety of devices which could provide overlapping data, and that's why it looks so precise IMO.