Support NeoGAF

[Refreshment.01]

Here's a question you might be able to answer, Skittzo. Earlier this year, Nintendo filed a patent which dealt with removable controllers which are non-electronic and whose button presses, etc. are "read" by an IR camera/sensor (link to the Gaf thread on it). While the joycons themselves are obviously electronic and communicate both via an electrical connection and bluetooth, it would seem like the other patent would be a natural fit for Switch, allowing them to sell additional custom controllers at a very low cost (potentially bundled with games). Reading through the patent, though, I don't see any reference to this sort of functionality.

My question is, effectively, if they were pursuing the IR controller tech, would it be expected that they would also include details of it in this patent (given that it appears to cover almost all functions of Switch's hardware)? Or, is the fact that they've already got a patent pending for it enough that they wouldn't include it here?

The fact that the feature is not described in this latest patent means it won't be included in the device that is launching March 2017. We already have precedents of similar final product patents with the Wii U and Wii.

However, the fact that the implementation of the IR camera doesn't match the previous patent doesn't mean the functionality won't be included in the Switch. If you look at my previous post, the latest patent cites using the IR camera in the right Joycon inside electronically simplified control shells which inputs are read in an optoelectrical way. As well as using the sensor for Wii like pointing, Biometrics, Pulse reading, Hand Gesture Recognition.

i remember doing some speculation of the advantages of having the Image Sensor on the detachable controllers:

http://m.neogaf.com/showpost.php?p=215867721

Instead of what people normally speculate about having diferent attachments. Nintendo could develop just two attachments, use the image senors inside them and let the user remove the top part of the controller with buttons...

http://m.neogaf.com/showpost.php?p=217280673

2. IRS in the detachable parts. Let's assume each detachable part is a Reverse Wii Remote in which the IR sensor points inward to measure button input. Even with the inclusion of Blutooth and IMS, the device becomes cheaper to produce than one with a more standard PCB.

But anyway, the reason why it is in the right Joycon is obvious: Flexibility.

• Given the Hybrid nature of the Swicth the IR camera in this position compliments the device better, since it can be used in more setup and scenarios. Like docked, when held in the hands or when set up with the kick stand.
• Modularity. Remember the Wii Remote? Well since a Joycon is even more smaller it's modularity is amplified. This means you can set inside shells with different form factors. These shells would be cheap and could use the Joycon buttons and stick as well as the IR camera to track Infrared reflective inputs.
• With the camera inside a Joycon, the user can point the sensor to any direction. This would have been far more limiting with the camera inside the Swicth's right part instead.

My mistake then, I didn't have time to read it over from the patent as I'm jumping between a lot of things.

Understandable, there's too much material to read.

The reason my dumb brain could pick up those details with a quick read was because many of the features described matched speculations i had in previous NX/Switch threads.

 

[Skittzo0413]

The fact that the feature is not described in this latest patent means it won't be included in the device that is launching March 2017. We already have precedents of similar final product patents with the Wii U and Wii.

This is not true at all. The patent application (filed in June, as a reminder) already has seen several major differences than what's in the product seen in the trailer.

From what we know so far:

• the dock shape is different
• the kickstand is in a different location
• the SD card slot is in a different location
• the volume buttons are a rocker rather than discrete buttons
• the sides of the Switch tablet have yet unidentified features not shown in any of the patent figures
• the vents on the bottom of the Switch tablet are in different locations
• the USB slots on the dock have not been detailed in the patent application (they are confirmed to exist, but the location isn't specified

And there is likely a lot more since we've barely seen the device in detail. While these might seem like minor features, the overall point is that there is quite a bit of breathing room between what's described in a patent and what is being sold- the real legal protection of the patent is described in the claims, and as long as the device doesn't stray too far from those claims it will get that legal protection.

So I wouldn't assume there can't be anything else in the final device besides what's described in the patent.

 

[Refreshment.01]

This is not true at all. The patent application (filed in June, as a reminder) already has seen several major differences than what's in the product seen in the trailer.

And there is likely a lot more since we've barely seen the device in detail. While these might seem like minor features, the overall point is that there is quite a bit of breathing room between what's described in a patent and what is being sold- the real legal protection of the patent is described in the claims, and as long as the device doesn't stray too far from those claims it will get that legal protection.

So I wouldn't assume there can't be anything else in the final device besides what's described in the patent.

In this case you are taking my post out of context and making assumptions of what im actually thinking or claiming. im aware of what you are saying and i fully agree with. That is why i gave the example of some of the "last/final" Wii U/Wii patents to avoid exactly what you are stating above, since going from memory, those patents got really close to the final products with just slight minor changes.

Remember the context of my post? It was about the position of the IR image sensor in the final Switch. i gave some logical reasons of why i think it was implemented that way. So that's what we should be discussing and not if i have at least a basic understanding of how patents work.

From what we know so far:

• the dock shape is different
• the kickstand is in a different location
• the SD card slot is in a different location
• the volume buttons are a rocker rather than discrete buttons
• the sides of the Switch tablet have yet unidentified features not shown in any of the patent figures
• the vents on the bottom of the Switch tablet are in different locations
• the USB slots on the dock have not been detailed in the patent application (they are confirmed to exist, but the location isn't specified

im fully aware from the changes you mention above. The patent filling is June (correct?), so it was been made even earlier than that and the October trailer was shot around September (i think according to one of the actors?) With so many months appart between these 2 events it makes sense to witness some changes and the possibility of the Switch in the trailer been even close to the final retail unit.

Different features have different priorities, the ones you mention are more in the back order and thus can be adjusted further down development than what i was discussing.

Here are some additional things to comment about as well as some other things that are not in your summary:

• Not only the craddle shape is different, but design alterations have been made so the user can only insert the Switch in one direction, even tought USB C is capable of work both ways, something the patent explains in detail about 2 potential docking modes. As we can see in the trailer with the "security" groove near the USB C port.
• Correct, the kickstand is in a different location and the entire mechanism is different. It wasn't even a kickstand in the first place but a rotating foot at the bottom of the device.
• About the highlighted and this is very important.This is something i think you are missing, is not that there are still unidentified features to the sides. The patent describes 2 Joycons and 2 Switch side designs, ones more refined than the others. The old Switch side diagram has just a lock mechanism and a rudimentary physical connection. On the other hand the new one has a more complex and reliable locking and communicating mechanism. That is the reason why i think you are claiming that there are still not unveiled features on the side of the Switch.
• The patent also have different Joycon designs. New ones have different LED sizes and spacing. Different shaped SL and SR buttons. Slightly variation of Synch button and a different shaped analog stick top part.
• Slightly related, there are also various Grips, includign the final one in terms of both design and features.
• The number of USB slots is detailed, 3 according to the patent. However, like you said no positioning described. This of course makes sense since the Craddle in the patent is old/place holder. However, we could speculate based on the video that the 3rd USB could be in the back since we can clearly see 2 lateral ones in the trailer.

Excuse any inconsistences i might had since im going from memory and i saw the patent last Friday. im not seen it now since i post with a 1GB mobile device than can't display large documents (and with lots of pictures to boot) correctly.

There was 1 or 2 things i was gonna point out but i totally forgot XD

Edit: Removed the 2 quotes that didn't belong to this post. Those are commented in the following one.

 

[Skittzo0413]

In this case you are taking my post out of context and making assumptions of what im actually thinking or claiming. im aware of what you are saying and i fully agree with. That is why i gave the example of some of the "last/final" Wii U/Wii patents to avoid exactly what you are stating above, since going from memory, those patents got really close to the final products with just slight minor changes.

Remember the context of my post? It was about the position of the IR image sensor in the final Switch. i gave some logical reasons of why i think it was implemented that way. So that's what we should be discussing and not if i have at least a basic understanding of how patents work.


im fully aware from the changes you mention above. The patent filling is June (correct?), so it was been made even earlier than that and the October trailer was shot around September (i think according to one of the actors?) With so many months appart between these 2 evens it makes sense to witness some changes and the possibility of the Switch in the trailer been even close to the final retail unit.

Different features have different priorities, the ones you mention are more in the back order and thus can be adjusted further down development than what i was discussing.

Here are some additional things to comment about as well as some other things that are not in your summary:

• Not only the craddle shape is different, but design alterations have been made so the user can only insert the Switch in one direction, even tought USB C is capable of work both ways, something the patent explains in detail about 2 potential docking modes. As we can see in the trailer with the "security" groove near the USB C port.
• Correct, the kickstand is in a different location and the entire mechanism is different. It wasn't even a kickstand in the first place but a rotating foot at the bottom of the device.
• About the highlighted and this is very important.This is something i think you are missing, Is not that there are unidentified features to the sides. The patent describes 2 Joycons and 2 Switch side designs, one more refined than the other. The old Switch side diagram has just a lock mechanism and a rudimentary phisical connection. On the other hand the new one has a more complex and reliable locking and communicating mechanism. That is the reason why i think you are claiming that there are still not unveiled features on the side of the Switch.
• The patent also have different Joycon designs. New ones have different LED sizes and spacing. Different shaped SL and SR buttons, slightly variation of Synch button and a different shaped analog stick top part.
• The number of USB slots is detailed, 3 according to the patent. However, like you said no positioning described. This of course makes sense since the Craddle in the patent is old/place holder. However, we could speculate based on the video that the 3rd USB could be in the back since we can clearly see 2 lateral ones in the trailer.

Excuse any inconsistences i might had since im going from memory and i saw the patent last Friday. im not seen it now since i post with a 1GB mobile device than can't display large documents (and with lots of pictures to boot) correctly.
STILL EDITING.

I must have misunderstood your post then, I apologize. I thought you were suggesting that the previous IR controller patent functionality could be included for attachments via the IR camera on the right joycon, but that it can't be done with IR sensors on the side of the Switch tablet portion since that's not seen in this patent.

If that's not what you were getting at then yeah ignore my post. I'll respond to your other points in a bit.

 

[Philippo]

Portable, home console, SD cards, Usb-C, customizable controls, great hardware (for what it is), VR, motion controls.

Sometimes more than a well enginered machine it looks like something that can do everything lol.

 

[Refreshment.01]

No Gear VR has been 720p, let that idea go. Those were 1080p or 1440p displays that were smaller than the switch. The pixel density is not comparable at all. Switch VR would need a better screen. Read some reports of people using the first Oculus prototypes. They were unusable for many many people due to the low res. Many people would get sick using them, it wasn't really until much later prototypes the device started to work acceptably for most users. It's safe to assume any Switch VR setup would need a revised setup with an improved display. Which is fine, it makes sense to have the patent now even if it's not something the current device is really capable of handling.

I don't think anyone expects the system to do VR right this instant. It's more future proofing as hardware advances and upgraded models and generational stuff.

Let's make various asumptions here: 1st, that indeed the Switch screen is 720P. And 2nd, that the screen artifacts are so apparent that Nintendo themselves judged that is not possible to offer a decent VR experience with that screen size and resolution.

But what about using the HMD as a personal viewer instead?

Used in this way there are less agressive transformations to be made to the image with the optics like is needed for VR, where the displayed image needs to be wraped around the user's FOV as much as posible. And also, since stereoscopy is not as neccesary, the system wouldn't incur in the performance penalties and reduced resolution of rendering 2 separate images per eye. Instead let's just say the function of the HMD is to project the image infront of the player.

There are various benefits i could see that extent to both presentation and gameplay:

• There's the possibility to project an image of an apparent bigger size than the screen's 6.2'' with a reduced screen door effect than what would happen if opting for "full VR"
• Isolating the player from his surrounding enviroment with the game's visual and sound can enhance games that rely on atmosphere. Again even if it's not "full VR".
• As for gameplay possibilities, since the Switch itself has IMUs, those can be used for relative measurement of the user's head positioning. Among the possibilties this allows, is camera control with head motion. There are some interesting interfaces developers can come up with, since there's the potential to track relative positioning of the 2 hands and the user's head with the Switchs default controls.

Think of it as a more natural extension of some control shemes Nintendo implemented in Wii and Wii U games. For example, imagine a shooter where both head movement and hand movement is used to aim, basically this will decouple the boundry system of Wii FPS, the head will move the camera around while the Joycon would control the cursor within the screen space. Another concrete example could be Star Fox Zero's control scheme, in which the head could control the aiming reticule just like the Fox's character visor does within the game.

i think the Switch weight and size is significantly reduced when the Joycons are detached to a point that the personal viewer function could be confortable enough for short gameplay sessions.i would relly like to know, is this would be atractive enough to anyone here.

 

[Pachimari]

So when it says SD cards, are we sure it'll be microSD?

 

[Aostia]

So when it says SD cards, are we sure it'll be microSD?

not yet, if I'm right

 

[CaviarMeths]

I would prefer SD for the higher capacity. With the average size of games presumably bigger than Wii U games, I can see myself blowing through a 256gb card in pretty short order.

 

[Refreshment.01]

Since the patent doesn't point out any IR emitters on the Swicth to be tracked by the right Joycon camera for pointing purposes (basically Wii Remote emulation), i was wondering about a less obvious way on how it could be used for emulating touch interfaces.

i 've mentioned that putting reflective markers in both the TV or Swicth screen could work assuning there's also an emitter within the right Joycon. However, i was thinking in a more closer type of emulation of the touch screen.

What if to enable the touch functionality when the Switch is in the cradle, the user lays the right Joycon on a flat surface, a table for example, with the camera pointing backwards instead of towards the Switch. Then the user moves his fingers close or away from the IR sensor to control a dot on the TV that moves in the X/Y coordinates. Then to register an input, the user touches the table.

There could be a cheap plastic accessory to simulate a trackpad. Imagine having a plastic surface that bends a little when preaure from a finger is applied. The right Joycon could be attached to it, have the user slide its finger on the surface to control the cursor and when the user applies preasure it registers touch. The Joycon can even vibrate the surface its attached to indicating an input has been registered, for more tactile feedback.

Wonder if Nintendo is considering that possibility? Given the patent descriptions about the applications of the IR sensor, i think the implementation i suggest here is within the realm of possibilites.

 

[Skittzo0413]

Since the patent doesn't point out any IR emitters on the Swicth to be tracked by the right Joycon camera for pointing purposes (basically Wii Remote emulation), i was wondering about a less obvious way on how it could be used for emulating touch interfaces.

i 've mentioned that putting reflective markers in both the TV or Swicth screen could work assuning there's also an emitter within the right Joycon. However, i was thinking in a more closer type of emulation of the touch screen.

What if to enable the touch functionality when the Switch is in the cradle, the user lays the right Joycon on a flat surface, a table for example, with the camera pointing backwards instead of towards the Switch. Then the user moves his fingers close or away from the IR sensor to control a dot on the TV that moves in the X/Y coordinates. Then to register an input, the user touches the table.

There could be a cheap plastic accessory to simulate a trackpad. Imagine having a plastic surface that bends a little when preaure from a finger is applied. The right Joycon could be attached to it, have the user slide its finger on the surface to control the cursor and when the user applies preasure it registers touch. The Joycon can even vibrate the surface its attached to indicating an input has been registered, for more tactile feedback.

Wonder if Nintendo is considering that possibility? Given the patent descriptions about the applications of the IR sensor, i think the implementation i suggest here is within the realm of possibilites.

That seems like a pretty complicated setup, and not something all that easily doable in a way where you have quick access to all your traditional button inputs. Of course, using the IR camera on the back of the Joycon with an emitter on the tablet also doesn't give you access to all those inputs...

I think the most obvious way to do it is like Skyward Sword's pointing mechanism- just use the gyro sensors. The only problem is how would you calibrate to the center of the screen, but isn't that doable just by pointing the Joycon at the center of the screen and pressing a button?

 

[Refreshment.01]

That seems like a pretty complicated setup, and not something all that easily doable in a way where you have quick access to all your traditional button inputs. Of course, using the IR camera on the back of the Joycon with an emitter on the tablet also doesn't give you access to all those inputs...

I think the most obvious way to do it is like Skyward Sword's pointing mechanism- just use the gyro sensors. The only problem is how would you calibrate to the center of the screen, but isn't that doable just by pointing the Joycon at the center of the screen and pressing a button?

You are confused Skittzo, im talking about emulating touch screen input, that is the actual touch screen.

Let me try to work it out with an example. Let's say fruit Ninja is ported to the Switch. When the user is in handheld mode, he slices the fruit using one or multiple fingers geturing directly on the touch screen.

When the device is docked, the user rests the right Joycon on a table with the IR window facing towards his direction. When the finger(s) are within the camera FOV a dot UI is represented on the TV screen signaling the relative position. Touching the surface of the table is registered as an input. The calibration process would be something to consider, that's why i mentioned a cheap plastic surface attachable to the Joycon as other alternative just in case, but i think is doable.

The setup is not complex at all and im leaving outside Wii games emulation because that actually has been talked in the past plenty, but still, we don't have anything conclusive from Nintendo in regards how they would handle the sensor bar emulation for such setup. Remember, no IR LED's to be found anywhere in the device as of now.

 

[Nostremitus]

You are confused Skittzo, im talking about emulating touch screen input, that is the actual touch screen.

Let me try to work it out with an example. Let's say fruit Ninja is ported to the Switch. When the user is in handheld mode, he slices the fruit using one or multiple fingers geturing directly on the touch screen.

When the device is docked, the user rests the right Joycon on a table with the IR window facing towards his direction. When the finger(s) are within the camera FOV a dot UI is represented on the TV screen signaling the relative position. Touching the surface of the table is registered as an input. The calibration process would be something to consider, that's why i mentioned a cheap plastic surface attachable to the Joycon as other alternative just in case, but i think is doable.

The setup is not complex at all and im leaving outside Wii games emulation because that actually has been talked in the past plenty, but still, we don't have anything conclusive from Nintendo in regards how they would handle the sensor bar emulation for such setup. Remember, no IR LED's to be found anywhere in the device as of now.

That's a lot more complicated than just using the Joycon as a Wii remote+...

 

[GaryD]

Is it possible the screen is 1080p, however the device renders at 720p and upscales to 1080? I'm just thinking for VR applications that might work better.

 

[Tom Nook Sawyer]

You are confused Skittzo, im talking about emulating touch screen input, that is the actual touch screen.

Let me try to work it out with an example. Let's say fruit Ninja is ported to the Switch. When the user is in handheld mode, he slices the fruit using one or multiple fingers geturing directly on the touch screen.

When the device is docked, the user rests the right Joycon on a table with the IR window facing towards his direction. When the finger(s) are within the camera FOV a dot UI is represented on the TV screen signaling the relative position. Touching the surface of the table is registered as an input. The calibration process would be something to consider, that's why i mentioned a cheap plastic surface attachable to the Joycon as other alternative just in case, but i think is doable.

The setup is not complex at all and im leaving outside Wii games emulation because that actually has been talked in the past plenty, but still, we don't have anything conclusive from Nintendo in regards how they would handle the sensor bar emulation for such setup. Remember, no IR LED's to be found anywhere in the device as of now.

Yeah, I'll put money on that not happening. Way too complicated for something that others have noted can be replicated far simpler with gyro controls. Plus you lose easy access to the right joy-con's buttons

 

[LordRaptor]

So when it says SD cards, are we sure it'll be microSD?

If you assume the patent diagrams are 'close enough' to final design to gauge approximate size from, it pretty much has to be MicroSD;

items 23 ('top' of device in exploded view) and 24 ('bottom' of device in exploded view) are for cartridge reader / external memory reader respectively.

 

[Skittzo0413]

You are confused Skittzo, im talking about emulating touch screen input, that is the actual touch screen.

Let me try to work it out with an example. Let's say fruit Ninja is ported to the Switch. When the user is in handheld mode, he slices the fruit using one or multiple fingers geturing directly on the touch screen.

When the device is docked, the user rests the right Joycon on a table with the IR window facing towards his direction. When the finger(s) are within the camera FOV a dot UI is represented on the TV screen signaling the relative position. Touching the surface of the table is registered as an input. The calibration process would be something to consider, that's why i mentioned a cheap plastic surface attachable to the Joycon as other alternative just in case, but i think is doable.

The setup is not complex at all and im leaving outside Wii games emulation because that actually has been talked in the past plenty, but still, we don't have anything conclusive from Nintendo in regards how they would handle the sensor bar emulation for such setup. Remember, no IR LED's to be found anywhere in the device as of now.

No, I understood what you were proposing and I know you meant this as a way to replicate touch screen functionality. I'm saying that resting a joycon on the table and doing it that way is needlessly complex, time consuming and it seems quite hard to switch between touch and traditional controls in that way. It also seems like it would require you to look (or at least glance) at the table taking your eyes of the screen every now and then.

If you do joycon pointing with the gyro sensor you can have a cursor appear on the screen and press or hold a button (like the trigger) to simulate a tap. The only issue with this is calibration (and re-calibration) but it really seems to be a much simpler setup than the one you were describing.

 

[Refreshment.01]

This is the third time explaining this. Let's try this again:

• As of right now, we don't have any confirmation of IR LED's or emitters in the Swicth going by the videos and patents.
• We don't see any LEDs in the craddle either so far. And it doesn't matter since the sensor bar equivalent would need to be in the main body to keep functionality consistent in any setup.
• No confirmation in the patent or rumors about the Switch handling Wii Remote type functionality with a reflective marker of some type.

So with the above points into consideration and talking about aproximating a multi touch screen functionality, i proposed the "inverted IR Sensor" functionality over a surface.

Yeah, I'll put money on that not happening. Way too complicated for something that others have noted can be replicated far simpler with gyro controls. Plus you lose easy access to the right joy-con's buttons

Please read the above explanation. And also gyro controls are a relative pointing method, for touch screen emulation a more absolute pointing method is better which is what the IR sensor provides.

Also is not complicated, laying the right Joycon over a flatsurface is not more complex than using a mouse.

That's a lot more complicated than just using the Joycon as a Wii remote+...

Is not about substituting Wii M+ functionality, is about emulating a multi touch screen.

A neat thing would be to include a plastic magnetic cover to protect the Swicth screen. The plastic cover would have a groove to attach the right Joycon to it and some surface markers. When not used for protecting purpose, the user attachs the right Joycon to it and it works like a very cheap trackpad for mobile type games. This also helps with calibration issues.

Hope this is explanation is a bit more clear.

EDIT.

No, I understood what you were proposing and I know you meant this as a way to replicate touch screen functionality. I'm saying that resting a joycon on the table and doing it that way is needlessly complex, time consuming and it seems quite hard to switch between touch and traditional controls in that way. It also seems like it would require you to look (or at least glance) at the table taking your eyes of the screen every now and then.

There's no need to glance at anything since the dot representing the finger on the TV gives the user all the necessay feedback. That's how a trackpad works, you don't glance at it occassionaly to use it.

If you do joycon pointing with the gyro sensor you can have a cursor appear on the screen and press or hold a button (like the trigger) to simulate a tap. The only issue with this is calibration (and re-calibration) but it really seems to be a much simpler setup than the one you were describing.

If the gyro was enough to emulate the touch screen, then the IR sensor wouldn't exist to emulate such functionality in the first place. It's the absolute positioning capabilty that helps for this specific use.

 

[calc84maniac]

Is it possible the screen is 1080p, however the device renders at 720p and upscales to 1080? I'm just thinking for VR applications that might work better.

Just having a higher resolution screen wouldn't make VR any better if you don't have the higher detail to go with it. You'd just be making everything blurry. And I mean everything, including non-VR games.

 

[Skittzo0413]

Think about playing Mario Maker with the control style you're describing- you have to place the joycon R on the table to be able to use the touch functionality, and then when you want to test your level you pick it up and start playing. Then if you need to make an edit, you have to set it down again and repeat. This is an incredibly complex and roundabout way of controlling a game. It could work fine for simpler touch-only games though.

Ideally, the best way to do this is to have a joycon grip-like accessory with a touch pad on it. But I'm sure they'll find a decent solution, especially since we have heard rumors about Mario Maker coming in some capacity.

There's no need to glance at anything since the dot representing the finger on the TV gives the user all the necessay feedback. That's how a trackpad works.


If the gyro was enough to emulate the touch screen, then the IR sensor wouldn't exist to emulate such functionality, yet it does. It's the absolute positioning capabilty that helps for this use.

I mean you'd have to glance at the table when you begin touching/hovering your finger, so that you line up correctly with the IR camera. Like I said above, this could work for touch only games but not for a lot of DS/3DS games or something like Mario Maker.

Gyro aiming seemed pretty precise in Skyward Sword, beyond the drifting/decalibration issues. I think it would work pretty well.

 

[Refreshment.01]

Think about playing Mario Maker with the control style you're describing- you have to place the joycon R on the table to be able to use the touch functionality, and then when you want to test your level you pick it up and start playing. Then if you need to make an edit, you have to set it down again and repeat. This is an incredibly complex and roundabout way of controlling a game. It could work fine for simpler touch-only games though.

Ideally, the best way to do this is to have a joycon grip-like accessory with a touch pad on it. But I'm sure they'll find a decent solution, especially since we have heard rumors about Mario Maker coming in some capacity.

Mario Maker as an example is quite inappropiate in this case. i was very clear im talking about mobile device type games were the touch screen is the only means of interaction and where the gesture timing is more strict.

For your Mario Maker example, the gyro to emulate mouse functionality is enough, as laying out elements to design a level is not dependant of timing or quick reflexes. Also your example is quite convulted. If interacting with Mario Maker using the right Joycon on the table, the player just uses the right hand for touch screen functionality and the left Joycon SNES style to test the level. But for MArio Maker, of course the Gyro or Right Joycon Wii Remote style would be enough, im not that stupid XD.

However to give a Mario example that falls in line with emulating touch screen functionality, the Boost Block mode in NSMBU is a closer comparison. Giving the frantic pace of this mode a quicker more absolute positioning method is necessary.

Other thing to consider is that laying the right Joycon on a surface can actually give you multi touch functionality that is dependant of position. For example, the camera in this way could track in space the positioning of the righ and left hand index fingers to register taps.

Don't get me wrong, i can see Nintendo not using this idea at all, however dismissing it as too complex or that gyro or IR poitng works better in all cases is not grasping the entire picture.

EDIT:

I mean you'd have to glance at the table when you begin touching/hovering your finger, so that you line up correctly with the IR camera. Like I said above, this could work for touch only games but not for a lot of DS/3DS games or something like Mario Maker.

Gyro aiming seemed pretty precise in Skyward Sword, beyond the drifting/decalibration issues. I think it would work pretty well.

That was my point since the begining "TOUCH ONLY" games mostly.

That depends on how far from your peripheral view the surfcae would be. But anyway, that is way i gave the screen cover example as a trackpad to simplify the setup substantially, give better feedback and it's rather cheap and plausible to implement.

 

[Skittzo0413]

Mario Maker as an example is quite inappropiate in this case. i was very clear im talking about mobile device type games were the touch screen is the only means of interaction and where the gesture timing is more strict.

For your Mario Maker example, the gyro to emulate mouse functionality is enough, as laying out elements to design a level is not dependant of timing or quick reflexes. Also your example is quite convulted. If interacting with Mario Maker using the right Joycon on the table, the player just uses the right hand for touch screen functionality and the left Joycon SNES style to test the level. But for MArio Maker, of course the Gyro or Right Joycon Wii Remote style would be enough, im not that stupid XD.

However to give a Mario example that falls in line with emulating touch screen functionality, the Boost Block mode in NSMBU is a closer comparison. Giving the frantic pace of this mode a quicker more absolute positioning method is necessary.

Other thing to consider is that laying the right Joycon on a surface can actually give you multi touch functionality that is dependant of position. For example, the camera in this way could track in space the positioning of the righ and left hand index fingers to register taps.

Don't get me wrong, i can see Nintendo not using this idea at all, however dismissing it as too complex or that gyro or IR poitng works better in all cases is not grasping the entire picture.

I don't really expect there to be many touch-only games on the Switch which is why I picked Mario Maker as the major example. Something like Boost Block mode in NSMBU could work with this, but I don't see why gyro pointing wouldn't work just as well there too.

As to the bolded, I definitely agree that this seems to be the only way I can think of to simulate multi-touch. I'm not sure how many games will require that though. I think part of the reason the Switch has a multi-touch screen is that they're simply so prevalent in the market that it's cheaper than going with a single touch screen. I doubt we'll see that many mobile games on the Switch to begin with, so I don't think this is a big focus of Nintendo's.

 

[Throwaway123456789]

[*]No confirmation in the patent or rumors about the Switch handling Wii Remote type functionality with a reflective marker of some type.[/list]
So with the above points into consideration and talking about aproximating a multi touch screen functionality, i proposed the "inverted IR Sensor" functionality over a surface.

It specifically mentions the use of the IR sensor as you would a Wii Remote, and specifically mentions it functioning with a reflective marker of some type:

[0494] In addition to (or instead of) identifying a gesture input, the infrared image-capturing section 123 may be used to calculate the position and/or the orientation of the right controller 4 with respect to a predetermined marker. For example, a user may place a marker at an intended position (e.g., around the display 12 or around the TV 6), and perform an operation of moving the right controller 4 within such a range that the infrared camera can capture an image of the marker. Note that the marker may be made of a material that reflects infrared light, for example. Then, the information processing device 1 can calculate the position and/or the orientation of the right controller 4 with respect to the marker based on the image-capturing results from the infrared image-capturing section 123. Moreover, the information processing device 1 can execute an information process by using the calculated information as a user's operation input.

 

[Huge Succeeded]

edit:nvm

 

[Refreshment.01]

Just having a higher resolution screen wouldn't make VR any better if you don't have the higher detail to go with it. You'd just be making everything blurry. And I mean everything, including non-VR games.

i think many are missing the point or not setting their expectations right about the prospect of VR on the Switch.

What you should expect is mobile type VR like GearVR or Daydream but with far superior controls. Something like the Wii Play series or short sessions of Animal Crossing (how amazing would be to visit other player houses in VR), these type of software seem like perfect examples. For 3D Videos, panoramas and movie watching too.

Sadly VR is were Pascal level GPU would have helped substantially, with Nvidia Simulataneous Multi Proyection taking some rendering burden off.

I don't really expect there to be many touch-only games on the Switch which is why I picked Mario Maker as the major example. Something like Boost Block mode in NSMBU could work with this, but I don't see why gyro pointing wouldn't work just as well there too.

That's a fair and sensible assumption but is not what we are discussing here. We are talking about a possible implementation of the IR sensor and what uses it might have.

As to the bolded, I definitely agree that this seems to be the only way I can think of to simulate multi-touch. I'm not sure how many games will require that though. I think part of the reason the Switch has a multi-touch screen is that they're simply so prevalent in the market that it's cheaper than going with a single touch screen. I doubt we'll see that many mobile games on the Switch to begin with, so I don't think this is a big focus of Nintendo's.

Same as above, is not a topic im discussing. Although since you brought it up, i do agree, that the ubiquity of multi touch screens these days has a role to play on it's inclusion. However, it does have gameplay possibilties that Nintendo could explore.

It specifically mentions the use of the IR sensor as you would a Wii Remote, and specifically mentions it functioning with a reflective marker of some type:

It is really hard for me to try covering every single possibility with each post one types. If you quote just one point of my post some of the meaning is lost, however maybe i should have explained my self better.

i do know that the patent mentions tracking the markers. im not lying here, as i pointed that out myself in this thread and mentioned the possibility in other occasions. Page 13:
http://m.neogaf.com/showpost.php?p=226681761

IR emitter embeded in the right Joycon with the camera itself (i think). That is why the camera can be used for:
a)To track markers, think Wii like pointing.
b)Biometric ID by measuring the blood vessel pattern of the hand.
c)Pulse detection.
d)Hand Gesture recognition.

e)ESPECULATION: By using reflective IR tape, the user could mark the exact corners of the TV screen and achieve 1:1 pointer tracking, basically a more exact Wii pointer capability.

But as explained many times, in this case the idea is to a offer closer touch screen functionality.

 

[Nostremitus]

This is the third time explaining this. Let's try this again:

• As of right now, we don't have any confirmation of IR LED's or emitters in the Swicth going by the videos and patents.
• We don't see any LEDs in the craddle either so far. And it doesn't matter since the sensor bar equivalent would need to be in the main body to keep functionality consistent in any setup.
• No confirmation in the patent or rumors about the Switch handling Wii Remote type functionality with a reflective marker of some type.

So with the above points into consideration and talking about aproximating a multi touch screen functionality, i proposed the "inverted IR Sensor" functionality over a surface.


Please read the above explanation. And also gyro controls are a relative pointing method, for touch screen emulation a more absolute pointing method is better which is what the IR sensor provides.

Also is not complicated, laying the right Joycon over a flatsurface is not more complex than using a mouse.

Is not about substituting Wii M+ functionality, is about emulating a multi touch screen.

A neat thing would be to include a plastic magnetic cover to protect the Swicth screen. The plastic cover would have a groove to attach the right Joycon to it and some surface markers. When not used for protecting purpose, the user attachs the right Joycon to it and it works like a very cheap trackpad for mobile type games. This also helps with calibration issues.

Hope this is explanation is a bit more clear.

EDIT.


There's no need to glance at anything since the dot representing the finger on the TV gives the user all the necessay feedback. That's how a trackpad works, you don't glance at it occassionaly to use it.


If the gyro was enough to emulate the touch screen, then the IR sensor wouldn't exist to emulate such functionality in the first place. It's the absolute positioning capabilty that helps for this specific use.

Without a screen to look at when you're touching it your idea would be really sloppy.

With a pointer on the screen you can look at the screen and see what you are about to interact with. Randomly touching a spot on a grey mat you aren't looking at and hoping it corresponds to the thing you see moving across a screen on the other side of the room is a terrible idea.

Try this, use screen cast and try to play a mobile game on your TV by setting your phone on the coffee table. Play the game without looking down at your phone in order to emulate having a gray mat with no screen.

 

[Skittzo0413]

Without a screen to look at when you're touching it your idea would be really sloppy.

With a pointer on the screen you can look at the screen and see what you are about to interact with. Randomly touching a spot on a grey mat you aren't looking at and hoping it corresponds to the thing you see moving across a screen on the other side of the room is a terrible idea.

Try this, use screen cast and try to play a mobile game on your TV by setting your phone on the coffee table. Play the game without looking down at your phone in order to emulate having a gray mat with no screen.

In the hypothetical described you would hover your finger over the table and the IR camera would be able to still read your finger location, and then display a cursor on the tv screen to show you where your finger is. Then you'd tap the table to simulate a tap.

I do agree that it seems a bit convoluted but it could work well for touch-only games, especially multi-touch games.

 

[Nostremitus]

In the hypothetical described you would hover your finger over the table and the IR camera would be able to still read your finger location, and then display a cursor on the tv screen to show you where your finger is. Then you'd tap the table to simulate a tap.

I do agree that it seems a bit convoluted but it could work well for touch-only games, especially multi-touch games.

You think a single camera in a Joycon would give you 3 dimensional tracking.... And not only that, but be able to tell which part of your hand should be tracked as the "pointers" without forcing everyone to wear what would essentially be a glove with mocap dots on the fingertips?

What are the specs for the joycon? Because now it needs it's own CPU, RAM, storage medium for firmware, etc to process this data as well as a larger battery to handle it.

 

[Skittzo0413]

You think a single camera in a Joycon would give you 3 dimensional tracking.... And not only that, but be able to tell which part of your hand should be tracked as the "pointers" without forcing everyone to wear what would essentially be a glove with mocap dots on the fingertips?

They have filed patents in the past actually demonstrating this, but you're right that it's not clear how well it would work in practice.

 

[Nostremitus]

They have filed patents in the past actually demonstrating this, but you're right that it's not clear how well it would work in practice.

The patent would have had a diagram of how the joycon 's CPU, RAM, Storage medium, and camera communicate to each other and then how that data is communicated to the Switch.

 

[Skittzo0413]

The patent would have had a diagram of how the joycon 's CPU, RAM, Storage medium, and camera communicate to each other and then how that data is communicated to the Switch.

No, I mean they have filed several patents about using IR cameras to detect gestures with hands/fingers. In general, not regarding the Switch specifically.

 

[Nostremitus]

No, I mean they have filed several patents about using IR cameras to detect gestures with hands/fingers. In general, not regarding the Switch specifically.

I know. I was saying that they would have mentioned it in the Switch patent if it was a thing they were doing

 

[Skittzo0413]

I know. I was saying that they would have mentioned it in the Switch patent if it was a thing they were doing

Ah, I see. They did mention gesture tracking with the IR camera on the joycon in the patent in the OP, so it's certainly possible they'll do something like that. There could also be several other unpublished patents relating to the Switch.

Plenty of room for other features to pop up.

 

[Nostremitus]

Ah, I see. They did mention gesture tracking with the IR camera on the joycon in the patent in the OP, so it's certainly possible they'll do something like that. There could also be several other unpublished patents relating to the Switch.

Plenty of room for other features to pop up.

Yes, but no mention on a joycon having it's own dedicated SoC, right?

It's pretty far fetched...

I'll gladly eat crow if it does materialize, but this is a terrible idea... Much better and easier to just use a pointer....

This has the added bonus of being able to sit back and use it from the couch.

 

[Skittzo0413]

Yes, but no mention on a joycon having it's own dedicated SoC, right?

It's pretty far fetched...

Oh I missed that edit of yours... why would that be necessary? Wouldn't the IR signal be transferred wirelessly to the console to do the necessary computation? Isn't that basically how the Wii works?

 

[Nostremitus]

Oh I missed that edit of yours... why would that be necessary? Wouldn't the IR signal be transferred wirelessly to the console to do the necessary computation? Isn't that basically how the Wii works?

For basic pointer controls there's not much data that needs to be crunched... Its just two dots and how far apart they move and whether they get bigger or not...


With what's being described, it has to Map out an area in 3 dimensional space, determine that a finger or fingers are present. Calculate their position within that space, and somehow determine when one of those fingers makes contacted with the table without the benefit of the laser array that wasn't in the patent either.

It's a lot more demanding. Sure, you could send the raw data to the Switch to process, but now that one joycon needs it's own dedicated 5ghz connection just like the gamepad... So that would also require another processor anyway...

Raw data is just that, raw.. big... Compressing it would also need a processor, and that would add lag.


Edit: Plus, gesture tracking using the IR sensor could just be tracking gestures made with the Joycon in hand... ie, tracking gestures the joycon makes. It would just mean it can do what the Wii Remote does.

 

[Refreshment.01]

I know. I was saying that they would have mentioned it in the Switch patent if it was a thing they were doing

So you find the idea intriguing enough to merit a reply, yet you don't read the following posts and with further explanations you keep missing the point?

Literally right above your post, there's a link describing a list of features from the patent that satisfies every single one of your requirements. Of course realistically speaking, this doesn't mean Nintendo would implement this tracking method but is something plausible if the patent claims about the Right Joycon sensor are indeed true.

"The relaxing while using the Joycon" comment get's me every time XD
i mean, i went through several replies explaining that this is not meant as a substitute for Gyro or Wii Remote IR cursor control. It is just describing another method of interaction that may suit certain types of games better and just like any of the other 2 methods, it has advanatages and disadvantages. Speaking of wich, this implementation could potentially allow to use an stylus over a surface for precise drawing. This would be doable in a cheap way with one that has an IR reflective tip or a LED controlled by an on/off switch withing the tool itself.

This diagram is in the original post maybe you find it interesting:

http://m.neogaf.com/view.php?img=https://abload.de/img/15178984-31ojstq.png

*nostremitus is the diagram displaying? im clicking it with this old device and for some reason im not seen it.

==================================================================
Edit: If i have the chance later, i will check out some IR sensor manufacturers to see if there's a component that could match the patent description at a reasonable price. Because i do understand your reservations about the capabilities of this device.

Other thing i wanted to mention since reading the patent the first time, is the distinction that takes place several times: "in this embodiment" and "in other embodiments". When the former is mentioned it seems to match features we deducted from the videos and rumors. With that in mind one could mistakenly assume what would end up in the final product.

 

[Refreshment.01]

Something to comment in the patent is the whole Joycon pairing process. It seems very simple, sliding each controller automatically synchs it to that Switch.

Another thing that comes to mind due to the various operation modes of the Joycons and the plethora of input methods in the Switch, is the possibility of context sensitive control schemes.

Taking a game such as Pikmin for example:

• When the game is played in Handheld mode the user can use the Touch Screen to more intuitevely interact with the game, for example tapping UI elements or using the "Go To" command by touching any point of a map. Using the Right Stick for cursor control since it's convenient when holding the unit with both hands and let's the user have a more relaxed position.
• When playing in console mode with the user holding a Joycon in each hand, the control scheme changes to Wii Remote type pointing controls. The cursor in this case is controlled with the Gyroscope pointing and is possible to map shortcut commands to gestures, for example, to scatter the Pikmin the user shakes the left Joycon.
• When attaching the Joycons to the Grip, then input mapping changes yet again. In this case the Right Stick can be used for camera control and the Gyroscope could be used to assist cursor aiming for a more granular and precise control.

This should be presented as an OPTION, since we are all aware how resistive some users can be when adapting to different control schemes. Never the less, these type of dynamic control schemes could be an interesting and unique feature offered by the Switch system. So, it would be nice if at least Nintendo took advantage of the potential.