Support NeoGAF

[bitbydeath]

As requested by ONQ.

Methods for interfacing with an interactive application using a controller with an integrated camera

Abstract
A computer implemented method for using a controller with at least one integrated camera is provided. The method includes identifying fixed points associated with objects disposed in a three-dimensional space using a depth camera that is part of with a controller. The method includes analyzing the identified fixed points captured by the depth camera and determining changes in position of the controller based on changes in position of the fixed points as determined from depth data captured by the depth camera. The method further includes triggering an input command when the controller is determined to in a particular positon based on the determined changes in position.

http://patft.uspto.gov/netacgi/nph-..."&OS="sony+interactive"&RS="sony+interactive"

Note: RuethnicCookie also stated the controller would have a camera included.

https://www.neogaf.com/threads/ruth...says-ps5-is-a-monster-details-inside.1468163/

 

[onQ123]

As requested by ONQ.

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=/netahtml/PTO/search-bool.html&r=2&f=G&l=50&co1=AND&d=PTXT&s1="sony+interactive"&OS="sony+interactive"&RS="sony+interactive"

Note: RuethnicCookie also stated the controller would have a camera included.

https://www.neogaf.com/threads/ruth...says-ps5-is-a-monster-details-inside.1468163/

Thanks and also Sony bought Softkinetic a few years ago & now have their own Depth Sensing company called 'Sony Depthsensing Solutions' https://www.sony-depthsensing.com/

 

[haxan7]

 

[Exentryk]

Camera in a controller would impact battery life pretty poorly, right?

 

[ultrazilla]

So basically part of Sony's PS5/VR 2.0 strategy.

 

[djshauny2]

Interesting stuff indeed. Wonder when we will see the PS5?

 

[zenspider]

Hate to be the one to ask... how is this different from Nintendo's Right Joy-con?

 

[Kagero]

I like it! The more crazy sauce, the better.

 

[DigitalScrap]

Please no. The light bar was bad enough.

 

[Grinchy]

So basically part of Sony's PS5/VR 2.0 strategy.

Yeah, for all we know, the patent picture shows the shape of a dualshock while the controller that actually uses this technology is really just some kind of VR controller that isn't the main one the system comes with.

 

[CRON]

If a simple trackpad and LED was responsible for giving the DualShock 4 a roughly 20 second battery life, I'm definitely not looking forward to how a (rumored) touch screen and camera will handicap the DualShock 5.

 

[onQ123]

Hate to be the one to ask... how is this different from Nintendo's Right Joy-con?

Being that this is just a patent / rumors at the moment how can anyone answer this?

 

[Iced Arcade]

I'm ok with it as long as they use a new battery technology. The currently controller battery is embarrassingly shit

 

[HeresJohnny]

Just make a box that plays games Sony, no need to go fucking up a good thing.

 

[Oddvintagechap]

This isn't a video camera. Basically helps the controller determine where it is in 3D space. Eliminates the need of a PS4 camera and glowing LEDs. Better tracking.

 

[DigitalScrap]

I'm ok with it as long as they use a new battery technology. The currently controller battery is embarrassingly shit

That's the problem. There is no new battery technology. They will have to make the DS5 the size of the Duke to get enough battery in there to make it last.

 

[Foxbat]

I didn't like kinect spying on me.

I really don't like a camera in my controller spying on me. Especially as I take it all over the house when I'm in a party chat.

No thanks Sony.

 

[Dener]

Sure

 

[Panajev2001a]

This isn't a video camera. Basically helps the controller determine where it is in 3D space. Eliminates the need of a PS4 camera and glowing LEDs. Better tracking.

It does not really, but reduces the need albeit with more chance of accumulated error on position tracking with s constantly moving reference point... it should be used with an external camera anyways.

 

[Ar¢tos]

I didn't like kinect spying on me.

I really don't like a camera in my controller spying on me. Especially as I take it all over the house when I'm in a party chat.

No thanks Sony.

Exactly how is a sensor measuring distances and positions between the player and the TV spying on you?
you are afraid it is a 4k hdr infrared camera pointing at your junk?

 

[Shpeshal Nick]

So they're putting Kinect in the controller.

That's sure to improve the battery life!

 

[ANIMAL1975]

Well, there goes my dreams of a selfies DS5....

 

[Foxbat]

Exactly how is a sensor measuring distances and positions between the player and the TV spying on you?
you are afraid it is a 4k hdr infrared camera pointing at your junk?

Looking at the OP, the diagram shows a depth camera, and an image capture.

So you're telling me there is no camera that can capture images?

 

[Whitecrow]

That would only drian batteyr life if it's always active and monitoring. Thng that we dont know yet : /

 

[Ar¢tos]

Looking at the OP, the diagram shows a depth camera, and an image capture.

So you're telling me there is no camera that can capture images?

Image capture =/= photo
First there would be no point having this sensor always active since it would eat away the battery. It probably will only be enabled when PSVR is active, and capturing image in "photo" would make the battery last nearly nothing. It's a machine, not a human. The data needed from a image to interpret depth, position, TV location etc doesn't involve the same color and light range a human eye does. We don't even know what resolution or the specs of the camera and you are already starting conspiracy theories.

 

[Darius87]

Please no. The light bar was bad enough.

How is that? you can make it dimmer or turn it off if you clever enough.

Just make a box that plays games Sony, no need to go fucking up a good thing.

if sony would listen people like you we would be stuck with:

 

[Ar¢tos]

Why people that don't care about VR won't simply ignore it instead of actively wasting their time trying to destroy it on social media?
You don't want VR? We get it. Nobody is forcing you to buy it.
Everytime something VR related is announced (or leaked) we get a parade of drive-by comments just bashing it.

 

[iconmaster]

It's probably for Depth Stranding.

(I'm here all week! Except Thanksgiving.)

 

[Horns]

Sounds like next gen controllers will be even more expensive.

 

[Thiagosc777]

Good, now the DS5 bateries will last 30 minutes.

/s

 

[Blam]

Welp sony's fucked lmao. Over ambitious shits gonna kill em.

 

[Racer!]

It does not really, but reduces the need albeit with more chance of accumulated error on position tracking with s constantly moving reference point... it should be used with an external camera anyways.

Assumptions on what grounds?

 

[Panajev2001a]

Assumptions on what grounds?

What other solution does not have the camera fixed and the controller/emitters moving and is able to do proper 1:1 3D positioning and movement with the kind of accuracy and position and lack of drift needed for VR? Wii's approach was clever but limited to the controller always having the IR emitter in the controller camera view "frustum"?

What were you thinking of?

 

[thelastword]

Exactly how is a sensor measuring distances and positions between the player and the TV spying on you?
you are afraid it is a 4k hdr infrared camera pointing at your junk?

I'm pretty sure this is console-wars bait, throwback to "kinect spying on me"....At MS's launch many complained about being spied on with Kinect, so it's his turn to do the same complaint with a Sony device...."in the name of balance of course"...

 

[Breakage]

This feels like Sony making the mistake of adding something that will end up having a detrimental effect in the long run.

 

[Zewp]

Please no more silly gimmicks, Sony. That oversized map button on your DS4 was really unnecessary and so was the shitty, low-quality speaker from the 90s.

Just give me a decent ergonomic controller with a battery that can last an entire Saturday.

 

[ThatStupidLion]

please dont measure the depth of my nostrils...
please dont measure the depth of my nostrils...
please dont measure the depth of my nostrils...

 

[Alx]

I'm a big fan of time-of-flight tech (shame that the new avatar shape is cropping my "better with kinect" banner ), but I'm not sure putting it in a regular controller is the best way to use it. It's great to put in a headset or a motion controller, where having a 1:1 3D tracking is important. But for the motion of a two-handed dualshock, inertial sensors are probably enough.

 

[Racer!]

Looking at the OP, the diagram shows a depth camera, and an image capture.

What other solution does not have the camera fixed and the controller/emitters moving and is able to do proper 1:1 3D positioning and movement with the kind of accuracy and position and lack of drift needed for VR? Wii's approach was clever but limited to the controller always having the IR emitter in the controller camera view "frustum"?

What were you thinking of?

Isnt this what wmr headsets does? And the upcoming Oculus Quest? The problem with those is that as soon as the controller is out of the headset sight, tracking breaks...which is probably why Sony wants a camera in DS5. The future of VR is computer vision inside out...thought everybody pretty much agreed on this.

 

[onQ123]

Just make a box that plays games Sony, no need to go fucking up a good thing.

I'm pretty sure that having a controller that can be tracked accurately in 3D space would make gaming a lot better than just sticking to button mashing.

How is that? you can make it dimmer or turn it off if you clever enough.

if sony would listen people like you we would be stuck with:

This

 

[onQ123]

I'm a big fan of time-of-flight tech (shame that the new avatar shape is cropping my "better with kinect" banner ), but I'm not sure putting it in a regular controller is the best way to use it. It's great to put in a headset or a motion controller, where having a 1:1 3D tracking is important. But for the motion of a two-handed dualshock, inertial sensors are probably enough.

I'm not sure if they will use it this way but having it in the controller would mean that you could sit the controller down & have close range hand tracking which would work better than a camera that's far away .

 

[yurinka]

I'm a big fan of time-of-flight tech (shame that the new avatar shape is cropping my "better with kinect" banner ), but I'm not sure putting it in a regular controller is the best way to use it. It's great to put in a headset or a motion controller, where having a 1:1 3D tracking is important. But for the motion of a two-handed dualshock, inertial sensors are probably enough.

Sony has another patent for split controller. I bet you'll be able to split the DS5 into two wands, each one with its own depth camera for better 1:1 3D tracking for VR. The good thing about having the camera in the controller instead of in the tv or the headset is that you'd be able to move freely in 3D without being constrained to the angle view and direction of the camera place in the TV or headset.

And if they also implement the touchscreen patent between these two wands you'd have a touchscreen to charge them + maybe using it for remote play and -plz- to play PSP and Vita games.

 

[EverydayBeast]

This thing will probably be PSVR2

 

[HeresJohnny]

I'm pretty sure that having a controller that can be tracked accurately in 3D space would make gaming a lot better than just sticking to button mashing.



This

People said the same thing about the Wiimote and it was sewage.

 

[onQ123]

People said the same thing about the Wiimote and it was sewage.

You're talking about gesture controls & not 3D mouse / pointer controls , Pointer controls are better than analog controls for aiming so having a controller with a depth camera + 9 degrees of freedom will open up doors for game play , like being able to lift your controller up & point down over a wall & shoot someone hiding behind it.

 

[FranXico]

You're talking about gesture controls & not 3D mouse / pointer controls , Pointer controls are better than analog controls for aiming so having a controller with a depth camera + 9 degrees of freedom will open up doors for game play , like being able to lift your controller up & point down over a wall & shoot someone hiding behind it.

I could swear I vaguely remember similar arguments made for the SIXAXIS back in the day.

It worked for a few games (I like Flower), but very few actually used the feature effectively.

 

[Racer!]

I could swear I vaguely remember similar arguments made for the SIXAXIS back in the day.

It worked for a few games (I like Flower), but very few actually used the feature effectively.

Im old enough to remember similar arguments made for the analog stick back in the day. That worked out pretty great. I`d rather they try new things than just offer the same over and over again. Some of it will work. Some of it wont.

 

[Elitist1945]

I'm not sure if they will use it this way but having it in the controller would mean that you could sit the controller down & have close range hand tracking which would work better than a camera that's far away .

I didn't imagine it that way but would make sense and be really cool.

Sony has another patent for split controller. I bet you'll be able to split the DS5 into two wands, each one with its own depth camera for better 1:1 3D tracking for VR. The good thing about having the camera in the controller instead of in the tv or the headset is that you'd be able to move freely in 3D without being constrained to the angle view and direction of the camera place in the TV or headset.

And if they also implement the touchscreen patent between these two wands you'd have a touchscreen to charge them + maybe using it for remote play and -plz- to play PSP and Vita games.

Forgot about the split controller patent. I honestly hope they do that, think it'd work really well.

 

[yurinka]

Camera in a controller would impact battery life pretty poorly, right?

Softkinetic has cameras that are basically the next gen of Kinect but with the size of a smartphone camera. So I'd say it may consume the same than a smartphone camera, which is very little.

 

[Alx]

Softkinetic has cameras that are basically the next gen of Kinect but with the size of a smartphone camera. So I'd say it may consume the same than a smartphone camera, which is very little.

It all depends on the range and framerate actually. Since it's an active system, that emits light to measure distances, it has to do more powerful flashes to see distant objects (otherwise the signal would be too weak when it gets back). The standard use for a depth camera embedded in a smartphone would be scanning the user's face, like in the iphoneX, so only a few centimeters away from the sensor. Doing inside-out tracking would require at least a few meters range. But then maybe they don't need to do it at high frequency, they could trigger IR blasts at low framerate and use it to correct drifting from motion sensors.