vpance
Member
That looks really accurate, not even considering its jerry rigged state.
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
Sony GDC eyetracking tech demo [using eyes to aim inside Infamous Second Son]
- Thread starter DieH@rd
- Start date
That looks really accurate, not even considering its jerry rigged state.
I can see limited use of this within the VR headset being useful, but it's use outside being even more useful than that. i.e. an AR setup. Use the AR to see the real world (perhaps even just your TV screen) and the eye tracking cameras are nice and close to your eyes to do their magic like we saw in the video.
Then bring the AR angle properly into the equation and supplement the view of your TV with AR - giving you screen a 3D overlay and providing illumiroom type stuff to - to fill out your vision and then all the eye tracking gubbins too on top of that... my brain just melted with the possibilities.
On second thoughts ignore everything I said - if they combined all this lot in a horror game with monsters hiding behind your own furniture, hearts would just fail!
Then bring the AR angle properly into the equation and supplement the view of your TV with AR - giving you screen a 3D overlay and providing illumiroom type stuff to - to fill out your vision and then all the eye tracking gubbins too on top of that... my brain just melted with the possibilities.
On second thoughts ignore everything I said - if they combined all this lot in a horror game with monsters hiding behind your own furniture, hearts would just fail!
Evolution of dating sims.
Dre_G_Writer
Member
GOT DAYUM SONY!!!
give a dude some time to calm down and take a breather. Dropping so many bombs, they gonna give me a heart attack.
Man, they are going all out this gen.
give a dude some time to calm down and take a breather. Dropping so many bombs, they gonna give me a heart attack.
Man, they are going all out this gen.
These are things I want from nextgen
Yeah, SCEJ actually did a glasses-mounted prototype for eyetracking a little while back with IR.
http://www.ijsei.com/papers/ijsei-10812-13.pdf
I think in an ideal implementation, you could do fantastic things with eyetracking in a HMD. But I'm not sure the latency performance is there yet. Have no idea, but they'd be extremely tight to do the really cool things anyway. Maybe it's good enough now to help augment aiming, like say, using eye tracking as well as head tracking to aim in EvE.
vpance
Member
Looks like the tracker is this model:
http://www.smivision.com/en/gaze-and-eye-tracking-systems/products/red-red250-red-500.html
Nice specs, but its for 60cm-80cm distance.
http://www.smivision.com/en/gaze-and-eye-tracking-systems/products/red-red250-red-500.html
Nice specs, but its for 60cm-80cm distance.
Rogue_Ledr
Banned
IF they can merge this tech into Morpheus, it would blow Oculus away.
It would be more Oculus than Oculus!
It would be more Oculus than Oculus!
Marc
Member
Wow, looks brilliant, usually it takes some getting used to a new technology but he is using it with ease.
I think I would prefer to control the camera with the right analogue and have the targeting reticule follow my eyes. Would find that less intrusive and more natural so your eyes are limited to the screen.
EDIT: Ooooh, just thought that strategy game would finally be very natural on consoles if this were incorporated,
I think I would prefer to control the camera with the right analogue and have the targeting reticule follow my eyes. Would find that less intrusive and more natural so your eyes are limited to the screen.
EDIT: Ooooh, just thought that strategy game would finally be very natural on consoles if this were incorporated,
MY EYES ARE UP HERE!
WeAreStarStuff
Member
Very impressive.
I don't see how it would strain your eyes. You're not doing anything extra, other than what your eyes would already be doing one way or the other when playing a game. The camera is what is doing the work, all you're doing is looking around.
Pretty much this.
That's impressive though, opens up so many opportunities.
If they include this in Morpheus, I don't think I'll be able to handle it...
I thought I was the only one that noticed. People want VR because they agree it's a more interactive experience... Yet they think auto-targeting wherever you look is a feature that complements that whole experience?
The video was nice too look at from a spectator's perspective, and I understand it's impossible to convey - via youtube - how this would work in VR without actual hands-on, but, I am certain that I would not enjoy THAT implementation (gameplay wise) in a VR game, which basically auto-aims wherever I look. What a way to kill the whole interactive VR experience!
Now give me VR, very limited eye-tracking(to simulate field of vision), with head-tracking, and Wiimote Plus/Move-style gun-controller, and drop me in the back alleys of a stunningly rendered Racoon City, and we're on to something special!
We'll get there, eventually...
so is it really tracking where you are looking or just where your eye balls are in space? At first I thought the former(such tech does exists) but it seems based on how he has to move his head around it looks to just be tracking how the eyes balls are moving with the head. Not necasarilly how the eyes rotate/look in their sockets. Which isnt nearly as impressive or useful.
While it is still useful on its own, it doesnt seem much different than say track IR except with the added benefit on not having to wear any tracking device.
While it is still useful on its own, it doesnt seem much different than say track IR except with the added benefit on not having to wear any tracking device.
I thought I was the only one that noticed. People want VR because they agree it's a more interactive experience... Yet they think auto-targeting wherever you look is a feature that complements that whole experience?
The video was nice too look at from a spectator's perspective, and I understand it's impossible to convey - via youtube - how this would work in VR without actual hands-on, but, I am certain that I would not enjoy THAT implementation (gameplay wise) in a VR game, which basically auto-aims wherever I look. What a way to kill the whole interactive VR experience!
Now give me VR, very limited eye-tracking(to simulate field of vision), with head-tracking, and Wiimote Plus/Move-style gun-controller, and drop me in the back alleys of a stunningly rendered Racoon City, and we're on to something special!
We'll get there, eventually...
I think there is a couple things here. Based on the video it is not necessarily tracking where you are looking exactly, just how your eyes in relation to your head are moving. Basically if you kept you head perfectly still, and looked around by rotating just your eyes, it wouldnt do anything or work.
It seems this same idea/tech could be replicated in VR by tieing your head rotations to control the camera. And to me that seems to be what will make VR useful for third person games like this, basically as a tool for perfect, or at least much more intuitive camera control. And could free up the second analog stick for some other function.
How I see it, its basically tracking your eye location in relation to each other(2 point track) to determine gaze, the same way the PS camera determines orientation of the Morphoues based on the blue LEDs on its outside . It doesn't seem to know where your eyes are actually looking independently though.
Graphics Horse
Member
Good question. Both techs exist but the way it's presented here looks like where your eyes are rather than your iris is. I'd have to try it for myself but what they show here doesn't look amazingly useful, as you've got to be looking to the side of the screen to rotate the camera, which isn't what you'd want to be doing in most situations. There are some great uses for eye tracking, like he says the attacks go where you're looking, that part is good, but I'm not sure that camera control should be put on top of that.
TheRealTalker
Banned
that was better and more fluid then I expected
Cthulhu_Steev
Member
You find looking at things tiring?
Graphics Horse
Member
What they're showing here is something that could cause your character to fall off a ledge simply because your eyes were pointing in slightly the wrong direction, it's possible that the amount of concentration you need to place on your eye movements isn't the most relaxing thing.
agreed, though I dont think this tech is necessarily tracking or detecting where you are actually looking.
I hadnt necessarily considered how eye tracking could benefit VR like you have, except for your last point. Ive always thought that stereo 3d wont really take off or be universally comfortable for people until they could really simulate true depth of field or eye focus. So that depending on where you are actually looking then you would get accurate focus change. Look at something close in the FG and the BG goes out of focus and vice versa, like with how are eyes work with real world depth perception.
Many people get queasy and headaches from stereo 3D because of things like this missing. Focus doesn't change when your brain thinks it should.
Graphics Horse
Member
figadapaura
Member
That's some pretty cool shit.
If this in incorporated in Morpheus, there's no way I'm not getting it.
If this in incorporated in Morpheus, there's no way I'm not getting it.
I noticed this as well but hopefully the tech will mature in the future where you can sit a little farther back. I have no problem with that distance when I play but, not everyone can sit that close.
nice, hadnt even considered using light feild to solve this problem, so does the implementation of how the light feild display is physically set up(prism like) just automatically control focus/defocus accuratly or does it simulate it by tracking your eyes? i would guess the former, but I am not famliar with light feild displays, I was only aware of the cameras and the semi interactive pictures.
SolidusDave
Member
The potential is immense (better immersion+ new gameplay options + saving rendering power), but if it's not miniaturized and/or built into the Morpheus prototype yet, I don't have hopes that we will see it in the actual release, unless that's even later than 2015 which would also suck
Graphics Horse
Member
It's physically set up to look right, so flexing your lens muscles will make it look different without the display changing, like a hologram. It probably needs many more images than they're using there to look perfect, I've not tried it myself.
That's actually pretty cool. Still the applications of the tech can't be used for every game but it already would be a step towards making a VR helmet a viable input device instead of mostly just a display device for traditional style games.
Too bad for Occulus Rift that Sony is throwing so much support behind this already, Sony will probably beat them to the punch with more infrastructure and support to get it on the ground running before the Occulus Rift even comes out the gate.
I still have my skepticism that VR will be as huge as people think but at this rate, we'll at least be able to see the tech capitalize on some of that potential even if it's just a modest success.
Too bad for Occulus Rift that Sony is throwing so much support behind this already, Sony will probably beat them to the punch with more infrastructure and support to get it on the ground running before the Occulus Rift even comes out the gate.
I still have my skepticism that VR will be as huge as people think but at this rate, we'll at least be able to see the tech capitalize on some of that potential even if it's just a modest success.
Pai Pai Master
Banned
I really don't think Sony sees it this way. OR and Morpheus are mostly targeting different demographics (platforms aside) but they can both benefit from increased exposure on both ends. There are already plenty of developers aware of and interested in Oculus Rift development and just like Sony's stance with bringing indies on board PS4, I figure they see OR's success as an opportunity for more devs/games on PS4 VR. They're not really in direct competition and I don't think Sony's in any hurry to add PC support and eat their piece of the pie.
nice, defintly sounds like a good direction for the the tech to go, wouldnt it also better mimic accurate light exposure(HDR) based on where you are looking?
dark_prinny
Banned
On rails Robocop VR shooter with eye targeting. I´ll buy that for a Dollar!
Graphics Horse
Member
I'm not sure about that, it would be interesting to see, could be applied to this Sony demo too. Ideally the display would get as bright as the sun and you could just let your eye handle it naturally again
Yep, when he wasn't actually firing. I'm talking about when you're in combat. Say some object catches your eye or you want to glance at your energy or something else. You don't want your fire to always track where you are looking.
On rails Robocop VR shooter with eye targeting. I´ll buy that for a Dollar!
I think that would be a nice gimmick, but to me that is the only use for eyetracking in gaming. Technology is old anyway, but surprisingly nobody who is not handicapped would control their mouse pointer in Windows with their eyes.
To move the view direction on screen to where the eyes look is totally counterintuitive as your eyes are on that spot anyway. So you move your eyes to your targeted area, then back to center then same again and so on.
For VR - OK, there might be some more gimmicky uses with the game knowing where you looking. But again that is counter-reality, as in real life you neither can move your gun with your eyes nor would the world around you usually react (or even know) on where you are looking.
This was awesome. I can't believe how natural it looks. It fits perfectly with the superhero genre.
One issue occurred to me though. I'm shooting a guy on the ground and an enemy pops out of a window, and I want to keep shooting at the guy on the ground while I identify what type of enemy is up top. What then?
One issue occurred to me though. I'm shooting a guy on the ground and an enemy pops out of a window, and I want to keep shooting at the guy on the ground while I identify what type of enemy is up top. What then?
vpance
Member
I think the express purpose of this tech is solely for Morpheus, which is exciting.
Sony's already said they're looking into eye tracking for VR, but the fact that they've gone out of their way to look at 3rd party solutions (SMI) and now a Sony engineer saying it could be miniaturized and used inside Morpheus tells me there's a good chance it'll make it into the retail product. They wouldn't bother to research and demo it to this extent otherwise. It's not like they'd release another camera with this tech to sit alongside the existing one. And every accessory so far has been created with the idea of eventually incorporating it into the entire VR experience.
Sony's already said they're looking into eye tracking for VR, but the fact that they've gone out of their way to look at 3rd party solutions (SMI) and now a Sony engineer saying it could be miniaturized and used inside Morpheus tells me there's a good chance it'll make it into the retail product. They wouldn't bother to research and demo it to this extent otherwise. It's not like they'd release another camera with this tech to sit alongside the existing one. And every accessory so far has been created with the idea of eventually incorporating it into the entire VR experience.
The VR application would be nice:
Focus / Defocus according to where you are looking at, simulating the natural DOF of the eye.. AAAND,
The VR setup could be further calibrated to detect convergence / divergence of the eyes, which would solve one of the most important problems of 3D and VR, This way you wouldn't have the convergence / divergence race of the eye when looking at things close by, which causes a lot of strain on the eyes and causes headaches.
I'm prone to the convergence / divergence problem a lot, this would be awesome.
Focus / Defocus according to where you are looking at, simulating the natural DOF of the eye.. AAAND,
The VR setup could be further calibrated to detect convergence / divergence of the eyes, which would solve one of the most important problems of 3D and VR, This way you wouldn't have the convergence / divergence race of the eye when looking at things close by, which causes a lot of strain on the eyes and causes headaches.
I'm prone to the convergence / divergence problem a lot, this would be awesome.
The very first thing in this thread is a link to 2 minutes of gameplay and impressions. And you think you're adding to the conversation by posting a Vine?
