• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
  • Hey Guest. Check out the NeoGAF 2.2 Update Thread for details on our new Giphy integration and other new features.

Sony GDC eyetracking tech demo [using eyes to aim inside Infamous Second Son]

DieH@rd

Banned
Dec 9, 2006
35,046
4
0
http://www.youtube.com/watch?v=kKYr9MaZw3I&hd=1


Sony's virtual reality headset already stacks up nicely against the Oculus Rift, but the company could also have a secret weapon. Last year, Sony revealed it was working on technology to let you control games with your eyes — technology that we just tried here at the 2014 Game Developers Conference. The same Sony team behind the Project Morpheus VR headset, the PlayStation Move, and the PlayStation Camera are also showing off a modified version of Infamous: Second Son where your eyes control where your character is looking.

With a special camera and software from SensoMotoric Instruments, setup is a breeze — you simply look at two different points on the screen — and then whenever you look at anything in the game, the camera rotates to center it in your view. We actually found it quicker and more accurate to aim at things than with the analog stick itself, no mean feat, and we were quickly wreaking havoc by throwing fireballs with pinpoint precision at traffic signs, trash cans, and the occasional enemy.

...

However, Sony's Eric Larsen says that the company believes it could actually miniaturize the components and bring down the price enough that eye-tracking could fit in a virtual reality headset like the Morpheus, which could create some exciting possibilities. In addition to controlling a character, eye-tracking inside a VR headset could theoretically be used to further enhance the player's sense of presence by faking depth of field, allowing your eyes to "focus" on virtual objects similarly to the way they focus in the real world. It also could allow game developers to explore mechanics where the player needs to avoid looking at certain things, like the bombs in the eye-tracking game of Fruit Ninja we played two years ago.
http://www.theverge.com/2014/3/20/5...s-with-your-eyes-be-the-future-of-playstation


 

UrbanRats

Member
Jun 25, 2009
40,855
1
830
If they tune this, it would be pretty incredible, especially in conjunction with Morpheus.
 

DieH@rd

Banned
Dec 9, 2006
35,046
4
0
Having very fast eyetracking integrated into Morpheus could allow not only new ways of controlling the games or giving games info about state of user [horror game that knows where you are looking could be so scary], but could also allow devs to create brand new engines.

Out eyes read high detail data at very small surface. Half of screen can be rendered with half details, and rest with even way less.
 

Ishan

Junior Member
Jun 13, 2013
5,050
0
425
this is interesting, hopefully you don't get motion sickness :p

actually eye tracking helps a lot. Think about how you naturally look in real life. We dont constantly weave and move our heads about. We made large motions with our heads but most of our focus and vision is predicated by where our eyes are facing. IMO good eye tracking is an important component of truly convenient consumer grade vr.
 

ScepticMatt

Member
May 14, 2011
4,094
2
745
Zurich
Eye tracking is the future of VR - it just makes too much sense.
* Foveated rendering, saving order(s) of magnitude in rendering power
* Eye-relative motion blur, removing stroboscopic artifacts ("1000Hz-feel")
* "real" depth of field
etc.
 

Ishan

Junior Member
Jun 13, 2013
5,050
0
425
Having very fast eyetracking integrated into Morpheus could allow not only new ways of controlling the games or giving games info about state of user [horror game that knows where you are looking could be so scary], but could also allow devs to create brand new engines.

Out eyes read high detail data at very small surface. Half of screen can be rendered with half details, and rest with even way less.

Also this. Things further in the periphery with eye tracking could be rendered worse. This should be included if they can manage it given the cost/size considerations.
 

glaurung

Member
Sep 14, 2005
13,092
0
0
39
Estonia
glaurung.planet.ee
Looks incredible.

I wonder what their distance from the detector should be. I mean the upper limit. Sitting next to a screen is nice, but I sit on a couch a far away from the TV.

If this is somehow combined with the VR headset, it's a game changer. Literally.
 

boeso

Member
Oct 28, 2013
952
0
0
Cool

Eye tracking is the future of VR - it just makes too much sense.
* Foveated rendering, saving order(s) of magnitude in rendering power
* Eye-relative motion blur, removing stroboscopic artifacts ("1000Hz-feel")
* "real" depth of field
etc.

COOOOL
 

Freeman

Banned
Aug 23, 2013
6,275
2,001
840
That was surprisingly impressive. The guy is playing second sons better than most videos I've seen so far.
I wonder the applications that a technology like this will have.
 
Jan 12, 2010
8,816
169
965
Maybe I'm not looking into this enough, but it doesn't seem like this would help in a first person virtual reality game. You don't want the image to center to your focus unless you turn your head. Maybe it could at least change the focus of the image? Or could be implemented in the UI, so if you look at something a mini HUD displays next to it with information?
 

GYNGA

Member
Jul 16, 2012
1,468
0
0
Maybe I'm not thinking into this enough, but it doesn't seem like this would help in a first person virtual reality game. You don't want the image to center to your focus unless you turn your head.
Eye tracking could be a big bonus for virtual reality if executed well, it's basically guaranteed that future VR headseats will have it
 

DieH@rd

Banned
Dec 9, 2006
35,046
4
0
Maybe I'm not looking into this enough, but it doesn't seem like this would help in a first person virtual reality game. You don't want the image to center to your focus unless you turn your head. Maybe it could at least change the focus of the image? Or could be implemented in the UI, so if you look at something a mini HUD displays next to it with information?

All that tracking data that Aiden can see in WatchDogs [names, professinos, trivia] in every pedestrian... Imagine if you can get that by simply marking NPCs with your eyes. That would remove a lot of clutter on the screen.
 

anchorman128

Member
Mar 13, 2012
557
0
0
CO
Here is a cool video of Sony PS4 eyetracking from GT around launch. They also talk about their partnership with NASA at the end, which is interesting as they just announced what that Mars deal with the Project Morpheus reveal. Also shows that as we all probably expected Magic Lab probably had a role in developing Sony's VR solutions.

https://www.youtube.com/watch?v=Jk3bjwazHsA

Twords they end, they show you a little bit of it working in Infamous, and how the eye tracking actually affects the game.
 

ashecitism

Member
Jun 29, 2013
17,989
22
0
Hungary
Maybe I'm not looking into this enough, but it doesn't seem like this would help in a first person virtual reality game. You don't want the image to center to your focus unless you turn your head. Maybe it could at least change the focus of the image? Or could be implemented in the UI, so if you look at something a mini HUD displays next to it with information?

It doesn't have to center the your focus, but only move the crosshair for ex

A couple of years ago Valve had a GDC presentation about bio-feedback and such and showed Portal 2 with eyetracking: https://www.youtube.com/watch?v=alQ8Ia808Jg

not the best of quality, but you can see it's the crosshair that is following the eyes
 
Aug 11, 2007
2,100
0
0
40
Ottawa
Maybe I'm not looking into this enough, but it doesn't seem like this would help in a first person virtual reality game. You don't want the image to center to your focus unless you turn your head. Maybe it could at least change the focus of the image? Or could be implemented in the UI, so if you look at something a mini HUD displays next to it with information?

Could it not be tethered to head tracking so that it just changes the focal point of a fixed image unless your head moves with your eyes? Similar to how the move has free movement within a box and you turn only when the move cursor hits the edges of that box. Except with eye tracking the edges if the box would be a hard limit and you would only turn if you move a controller or your head.
 

anchorman128

Member
Mar 13, 2012
557
0
0
CO
Maybe I'm not looking into this enough, but it doesn't seem like this would help in a first person virtual reality game. You don't want the image to center to your focus unless you turn your head. Maybe it could at least change the focus of the image? Or could be implemented in the UI, so if you look at something a mini HUD displays next to it with information?

Watch the video I posted above, it not only shows Sony's PS4 eyetracking in motion. But also shows Geoff Keighley playing Infamous : SS with eyetracking for a moment and explains the various ways they can use it.
 

lonely

Member
May 10, 2013
591
0
0
If developers and console makers put stuff like this in native it would really help out charities like Special Effect. That's what's most exciting about this.
 
Jan 12, 2010
8,816
169
965
Watch the video I posted above, it not only shows Sony's PS4 eyetracking in motion. But also shows Geoff Keighley playing Infamous : SS with eyetracking for a moment and explains the various ways they can use it.

Cool, thanks. Will have to watch later with sound, at work!
 

Sorc3r3r

Member
Jun 6, 2007
1,568
121
1,195
In my opinion this is something that must be inside the final headset.
I think it will be one of the pillar of the VR.
 

Betta Lines

Member
Feb 13, 2010
9,734
0
0
twitter.com
I'm not sure how this would work for consoles (apart from being integrated into VR). That camera was really securely taped down, and the guy had to sit very still. That's fine sat at a desk, but consoles are much more randomly placed in a bedroom or living space. I very much doubt the PS4 Camera has the accuracy to do this, so it would require yet another peripheral to work with a TV.

I can't see it going into the VR headset (at least not in the PS4 version) if they want to make the thing remotely affordable.