One of the big challenges facing VR is how to convey the experience through video. This green screen technique is an effective way to help bridge that gap.
(these have been mentioned in the other VR threads - I just wanted to start this one to gather the current examples and any new ones that come up as more developers try it out)
Fantastic Contraption
(full video)
When a third Lighthouse controller is attached to a camera for tracking, and the game is configured to output a video stream from that controller's perspective, they can use green screen to combine the camera's feed with the render and show the player in the VR scene like this.
The developers have included a way for people to do this themselves and posted a guide for it here:
http://northwaygames.com/how-to-mixed-reality/
This do-it-yourself method doesn't include a third Lighthouse controller, so it relies on a fixed camera and manually calibrating the game to match it.
Here's an earlier video using this setup:
Mixed reality introduction to Fantastic Contraption (15 minutes)
Audioshield
(full video)
Job Simulator
(test video)
They're also experimenting with different camera methods in "spectator mode", where the streamer can spawn a virtual camcorder in-game and place it wherever they want, then step in front of the virtual camera:
Spectator mode (2:21)
Impressive that this camcorder also includes a preview feed on its display!
That smoothing feature should be really helpful for those streams where the player is looking around a bit too quickly and it's hard to follow.
Other games soon?
Valve has created a Unity plugin for using the tracked-camera technique in SteamVR, so hopefully we'll see more of these from other developers soon. There's an active thread for it here:
https://steamcommunity.com/app/358720/discussions/0/405694031549662100/
This could work for external cameras in other genres too - for example, it should be fine for showing the player in the driver's seat in a racing game. The camera's position would just be relative to the driver's seat as the car moved around the track.
(these have been mentioned in the other VR threads - I just wanted to start this one to gather the current examples and any new ones that come up as more developers try it out)
Fantastic Contraption
(full video)
When a third Lighthouse controller is attached to a camera for tracking, and the game is configured to output a video stream from that controller's perspective, they can use green screen to combine the camera's feed with the render and show the player in the VR scene like this.
The developers have included a way for people to do this themselves and posted a guide for it here:
http://northwaygames.com/how-to-mixed-reality/
This do-it-yourself method doesn't include a third Lighthouse controller, so it relies on a fixed camera and manually calibrating the game to match it.
Here's an earlier video using this setup:
Mixed reality introduction to Fantastic Contraption (15 minutes)
Audioshield
(full video)
Job Simulator
(test video)
They're also experimenting with different camera methods in "spectator mode", where the streamer can spawn a virtual camcorder in-game and place it wherever they want, then step in front of the virtual camera:
Spectator mode (2:21)
Impressive that this camcorder also includes a preview feed on its display!
That smoothing feature should be really helpful for those streams where the player is looking around a bit too quickly and it's hard to follow.
Other games soon?
Valve has created a Unity plugin for using the tracked-camera technique in SteamVR, so hopefully we'll see more of these from other developers soon. There's an active thread for it here:
https://steamcommunity.com/app/358720/discussions/0/405694031549662100/
Valve said:
This could work for external cameras in other genres too - for example, it should be fine for showing the player in the driver's seat in a racing game. The camera's position would just be relative to the driver's seat as the car moved around the track.
