• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Sony patents Depth Camera within a controller

bitbydeath

Member
As requested by ONQ.

Methods for interfacing with an interactive application using a controller with an integrated camera

Abstract
A computer implemented method for using a controller with at least one integrated camera is provided. The method includes identifying fixed points associated with objects disposed in a three-dimensional space using a depth camera that is part of with a controller. The method includes analyzing the identified fixed points captured by the depth camera and determining changes in position of the controller based on changes in position of the fixed points as determined from depth data captured by the depth camera. The method further includes triggering an input command when the controller is determined to in a particular positon based on the determined changes in position.



http://patft.uspto.gov/netacgi/nph-..."&OS="sony+interactive"&RS="sony+interactive"

Note: RuethnicCookie also stated the controller would have a camera included.

https://www.neogaf.com/threads/ruth...says-ps5-is-a-monster-details-inside.1468163/
 

onQ123

Member


Thanks and also Sony bought Softkinetic a few years ago & now have their own Depth Sensing company called 'Sony Depthsensing Solutions' https://www.sony-depthsensing.com/
 
Last edited:

Grinchy

Banned
So basically part of Sony's PS5/VR 2.0 strategy.
Yeah, for all we know, the patent picture shows the shape of a dualshock while the controller that actually uses this technology is really just some kind of VR controller that isn't the main one the system comes with.
 

CRON

Banned
If a simple trackpad and LED was responsible for giving the DualShock 4 a roughly 20 second battery life, I'm definitely not looking forward to how a (rumored) touch screen and camera will handicap the DualShock 5.
 
This isn’t a video camera. Basically helps the controller determine where it is in 3D space. Eliminates the need of a PS4 camera and glowing LEDs. Better tracking.
 

DigitalScrap

Gold Member
I'm ok with it as long as they use a new battery technology. The currently controller battery is embarrassingly shit

That's the problem. There is no new battery technology. They will have to make the DS5 the size of the Duke to get enough battery in there to make it last.
 

Foxbat

Banned
I didn't like kinect spying on me.

I really don't like a camera in my controller spying on me. Especially as I take it all over the house when I'm in a party chat.

No thanks Sony.
 

Panajev2001a

GAF's Pleasant Genius
This isn’t a video camera. Basically helps the controller determine where it is in 3D space. Eliminates the need of a PS4 camera and glowing LEDs. Better tracking.

It does not really, but reduces the need albeit with more chance of accumulated error on position tracking with s constantly moving reference point... it should be used with an external camera anyways.
 

Ar¢tos

Member
I didn't like kinect spying on me.

I really don't like a camera in my controller spying on me. Especially as I take it all over the house when I'm in a party chat.

No thanks Sony.
Exactly how is a sensor measuring distances and positions between the player and the TV spying on you?
you are afraid it is a 4k hdr infrared camera pointing at your junk?
 

Foxbat

Banned
Exactly how is a sensor measuring distances and positions between the player and the TV spying on you?
you are afraid it is a 4k hdr infrared camera pointing at your junk?

Looking at the OP, the diagram shows a depth camera, and an image capture.

So you're telling me there is no camera that can capture images?
 

Ar¢tos

Member
Looking at the OP, the diagram shows a depth camera, and an image capture.

So you're telling me there is no camera that can capture images?
Image capture =/= photo
First there would be no point having this sensor always active since it would eat away the battery. It probably will only be enabled when PSVR is active, and capturing image in "photo" would make the battery last nearly nothing. It's a machine, not a human. The data needed from a image to interpret depth, position, TV location etc doesn't involve the same color and light range a human eye does. We don't even know what resolution or the specs of the camera and you are already starting conspiracy theories.
 

Ar¢tos

Member
Why people that don't care about VR won't simply ignore it instead of actively wasting their time trying to destroy it on social media?
You don't want VR? We get it. Nobody is forcing you to buy it.
Everytime something VR related is announced (or leaked) we get a parade of drive-by comments just bashing it.
 
Last edited:

Horns

I drop hot takes hoping you'll argue with me. Just ignore me.
Sounds like next gen controllers will be even more expensive.
 

Panajev2001a

GAF's Pleasant Genius
Assumptions on what grounds?

What other solution does not have the camera fixed and the controller/emitters moving and is able to do proper 1:1 3D positioning and movement with the kind of accuracy and position and lack of drift needed for VR? Wii’s approach was clever but limited to the controller always having the IR emitter in the controller camera view “frustum”?

What were you thinking of?
 

thelastword

Banned
Exactly how is a sensor measuring distances and positions between the player and the TV spying on you?
you are afraid it is a 4k hdr infrared camera pointing at your junk?
I'm pretty sure this is console-wars bait, throwback to "kinect spying on me"....At MS's launch many complained about being spied on with Kinect, so it's his turn to do the same complaint with a Sony device...."in the name of balance of course"...
 

Breakage

Member
This feels like Sony making the mistake of adding something that will end up having a detrimental effect in the long run.
 

Zewp

Member
Please no more silly gimmicks, Sony. That oversized map button on your DS4 was really unnecessary and so was the shitty, low-quality speaker from the 90s.

Just give me a decent ergonomic controller with a battery that can last an entire Saturday.
 

DadEggs

Member
please dont measure the depth of my nostrils...
please dont measure the depth of my nostrils...
please dont measure the depth of my nostrils...
 

Alx

Member
I'm a big fan of time-of-flight tech (shame that the new avatar shape is cropping my "better with kinect" banner :p), but I'm not sure putting it in a regular controller is the best way to use it. It's great to put in a headset or a motion controller, where having a 1:1 3D tracking is important. But for the motion of a two-handed dualshock, inertial sensors are probably enough.
 
Last edited:

Racer!

Member
Looking at the OP, the diagram shows a depth camera, and an image capture.
What other solution does not have the camera fixed and the controller/emitters moving and is able to do proper 1:1 3D positioning and movement with the kind of accuracy and position and lack of drift needed for VR? Wii’s approach was clever but limited to the controller always having the IR emitter in the controller camera view “frustum”?

What were you thinking of?

Isnt this what wmr headsets does? And the upcoming Oculus Quest? The problem with those is that as soon as the controller is out of the headset sight, tracking breaks...which is probably why Sony wants a camera in DS5. The future of VR is computer vision inside out...thought everybody pretty much agreed on this.
 
Last edited:

onQ123

Member
I'm a big fan of time-of-flight tech (shame that the new avatar shape is cropping my "better with kinect" banner :p), but I'm not sure putting it in a regular controller is the best way to use it. It's great to put in a headset or a motion controller, where having a 1:1 3D tracking is important. But for the motion of a two-handed dualshock, inertial sensors are probably enough.

I'm not sure if they will use it this way but having it in the controller would mean that you could sit the controller down & have close range hand tracking which would work better than a camera that's far away .
 

yurinka

Member
I'm a big fan of time-of-flight tech (shame that the new avatar shape is cropping my "better with kinect" banner :p), but I'm not sure putting it in a regular controller is the best way to use it. It's great to put in a headset or a motion controller, where having a 1:1 3D tracking is important. But for the motion of a two-handed dualshock, inertial sensors are probably enough.
Sony has another patent for split controller. I bet you'll be able to split the DS5 into two wands, each one with its own depth camera for better 1:1 3D tracking for VR. The good thing about having the camera in the controller instead of in the tv or the headset is that you'd be able to move freely in 3D without being constrained to the angle view and direction of the camera place in the TV or headset.

And if they also implement the touchscreen patent between these two wands you'd have a touchscreen to charge them + maybe using it for remote play and -plz- to play PSP and Vita games.
 
Last edited:

onQ123

Member
People said the same thing about the Wiimote and it was sewage.

You're talking about gesture controls & not 3D mouse / pointer controls , Pointer controls are better than analog controls for aiming so having a controller with a depth camera + 9 degrees of freedom will open up doors for game play , like being able to lift your controller up & point down over a wall & shoot someone hiding behind it.
 

FranXico

Member
You're talking about gesture controls & not 3D mouse / pointer controls , Pointer controls are better than analog controls for aiming so having a controller with a depth camera + 9 degrees of freedom will open up doors for game play , like being able to lift your controller up & point down over a wall & shoot someone hiding behind it.
I could swear I vaguely remember similar arguments made for the SIXAXIS back in the day.

It worked for a few games (I like Flower), but very few actually used the feature effectively.
 

Racer!

Member
I could swear I vaguely remember similar arguments made for the SIXAXIS back in the day.

It worked for a few games (I like Flower), but very few actually used the feature effectively.

Im old enough to remember similar arguments made for the analog stick back in the day. That worked out pretty great. I`d rather they try new things than just offer the same over and over again. Some of it will work. Some of it wont.
 

Elitist1945

Member
I'm not sure if they will use it this way but having it in the controller would mean that you could sit the controller down & have close range hand tracking which would work better than a camera that's far away .

I didn't imagine it that way but would make sense and be really cool.

Sony has another patent for split controller. I bet you'll be able to split the DS5 into two wands, each one with its own depth camera for better 1:1 3D tracking for VR. The good thing about having the camera in the controller instead of in the tv or the headset is that you'd be able to move freely in 3D without being constrained to the angle view and direction of the camera place in the TV or headset.

And if they also implement the touchscreen patent between these two wands you'd have a touchscreen to charge them + maybe using it for remote play and -plz- to play PSP and Vita games.

Forgot about the split controller patent. I honestly hope they do that, think it'd work really well.
 

yurinka

Member
Camera in a controller would impact battery life pretty poorly, right?
Softkinetic has cameras that are basically the next gen of Kinect but with the size of a smartphone camera. So I'd say it may consume the same than a smartphone camera, which is very little.
 

Alx

Member
Softkinetic has cameras that are basically the next gen of Kinect but with the size of a smartphone camera. So I'd say it may consume the same than a smartphone camera, which is very little.

It all depends on the range and framerate actually. Since it's an active system, that emits light to measure distances, it has to do more powerful flashes to see distant objects (otherwise the signal would be too weak when it gets back). The standard use for a depth camera embedded in a smartphone would be scanning the user's face, like in the iphoneX, so only a few centimeters away from the sensor. Doing inside-out tracking would require at least a few meters range. But then maybe they don't need to do it at high frequency, they could trigger IR blasts at low framerate and use it to correct drifting from motion sensors.
 
Top Bottom