• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Nintendo(?) Patent Application - Handheld with outward-facing linear image sensors

The patent application odyssey continues.

Like the last one, the assignee/applicant on this one is not Nintendo but one of the patent application inventors, Fumihiko Inoue.

So how do we know it's/he's related to Nintendo?

Well, a bit of researching returned an interview with him on Ninendo's japanese site (here), and he's one of the inventors on a couple of other -older- patents (here and here) as well, for which the assignee was indeed Nintendo.

There are actually many other patents with an inventor with the same name (is it him?). but they all have "Spansion LLC" (company that designs flash memory, microcontrollers, mixed-signal and analog products), as the assignee.

As you'll see below, this is clearly a gaming device-related patent application, one that even shows an Amiibo-like figure in one example.

It might not be as exciting as the rotary-shoulder-buttons application from a couple of days ago, but it's still interesting.

Does it have something to do with the NX/next handheld? Who knows.

United States Patent Application 20150253932

Filed: 02/25/2015
Published: 09/10/2015

Abstract
An example system includes a housing having at least one surface provided with a display, a linear image sensor located, facing outside the housing, at a side part or a corner part of the housing when the surface provided with the display is viewed as a front surface, and an information processing unit performing information processing based on an image obtained by the linear image sensor.

fig11np92.jpg


In the game apparatus 1 according to the present example embodiment, a linear image sensor 5 is disposed at each of four side parts (side surfaces) of the housing 2 so as to face outward. That is, the linear image sensor 5 is located at each of the side parts of the housing 2 when the surface provided with the display 3 is viewed as the front surface. Moreover, the linear image sensor 5 is oriented in the direction along the surface provided with the display surface 3. Though only two linear image sensors 5 are illustrated in FIG. 1, the game apparatus 1 includes four linear image sensors 5. Each side surface of the housing 2 has a substantially rectangular elongated shape. Each linear image sensor 5 has a long linear or rectangular shape, and is arranged in the longitudinal direction of each side surface. The linear image sensor 5 is an imaging device which utilizes a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) or the like. The linear image sensor 5 is a device used for reading a document, for example, in a scanner or the like. The linear image sensor 5 is also referred to as, for example, a line image sensor, a one-dimensional image sensor or the like.

Space Pointer Function

fig3pzp9p.jpg


FIG. 3 shows an example non-limiting schematic view for illustrating a space pointer function of a game apparatus;

The game apparatus 1 according to the present example embodiment obtains an image by the linear image sensor 5 located on each of the side surfaces on the housing 2 taking an image of the side part. This allows the game apparatus 1 to use the peripheral region of the housing 2 as an acceptance region for user operations. The user of the game apparatus 1 places the game apparatus 1, for example, on a flat desk or the like. The user may perform an operation of indicating (pointing) the peripheral region of the housing 2 of the game apparatus 1 with a finger or the like to perform, for example, an operation for a game. In the present example embodiment, this function of the game apparatus 1 is referred to as a space pointer function.

It is not always necessary for the game apparatus 1 to be placed on a desk or the like to implement the space pointer function. For example, it is also possible for the user to hold the game apparatus 1 with one hand and to perform an operation in the peripheral region with the other hand. The space pointer function of the game apparatus 1 may be implemented in any arbitrary location in a space FIG. 3 shows an example non-limiting schematic view for illustrating the space pointer function of the game apparatus 1. When, for example, the game apparatus 1 is placed on a flat desk or the like, the linear image sensor 5 located on a side surface of the housing 2 takes an image in a direction substantially parallel to the desk surface, from the side surface of the housing 2 outward, i.e. in the direction along the desk surface. Thus, the imaging range of the linear image sensor 5 is a substantially rectangular range from the side surface of the housing 2 to a part which is distant from the side surface by a predetermined distance. The regions indicated by broken lines in FIG. 3 are to be operation acceptance regions 51 in the space pointer function. The infrared light source 6 of the game apparatus 1 emits infrared light so that at least the inside areas of the operation acceptance regions 51 are irradiated with the infrared light. Since the game apparatus 1 is provided with linear image sensors 5 respectively on four side surfaces of the housing 2, the operation acceptance regions 51 are located at four sides of the housings 2.

Side Surface Touching Operation

fig7k9pwv.jpg


FIG. 7 shows an example non-limiting schematic view for illustrating a side surface touching operation in a game apparatus;

The game apparatus 1 according to the present example embodiment uses the linear image sensor 5 provided on a side surface of the housing 2 to accept an operation of approaching or making contact with a side surface of the housing 2. In the present example embodiment, a side surface touching operation includes a case where a finger or the like of a user actually makes contact with a side surface of the housing 2, and also a case where a finger or the like is placed close to a side surface within a certain distance therefrom without actually being in contact with the side surface. FIG. 7 shows an example non-limiting schematic view for illustrating a side surface touching operation in the game apparatus 1.

Figure Detection Function

fig8gmqx8.jpg


FIG. 8 shows an example non-limiting schematic view for illustrating a figure detecting function of a game apparatus;
The game apparatus 1 according to the present example embodiment implements a game also using a specific figure associated with a specific game program 91. FIG. 8 shows an example non-limiting schematic view for illustrating a figure detecting function of the game apparatus 1. In the illustrated example, a FIG. 60 representing a character of a mouse is used in a game. The FIG. 60 is, for example, a molded piece made of synthetic resin. The FIG. 60 is provided together with the recording medium 9 in which the game program 91 for implementing a game using the FIG. 60 is recorded, or provided independently from the recording medium 9.

Use Mode Determining Function

fig12z4pce.jpg


FIG. 12 shows an example non-limiting schematic view for illustrating a use mode determining function of a game apparatus;
The game apparatus 1 according to the present example embodiment has a function of determining which one of a horizontal posture and a vertical posture the housing 2 has when the user holds the housing 2 during use. In the present example embodiment, this function of the game apparatus 1 is referred to as a use mode determining function. FIG. 12 shows an example non-limiting schematic view for illustrating a use mode determining function of the game apparatus 1. The top part of FIG. 12 illustrates a state where the user is holding the game apparatus 1 to be oriented in the horizontally-long direction. The bottom part of FIG. 12 illustrate a state where the user has changed the way of holding the game apparatus 1 and is now holding it to be oriented in the vertically-long direction. On the display 3 of the game apparatus 1, a menu screen is displayed as an example. On the menu screen in the present example, items for selection such as game selection, screen setting, sound setting and the like are displayed to be vertically aligned with one another. On the menu screen, the user can operate the operation unit 4 to select any of the items.

The determination on the mode of use by the acceleration sensor 15 may have a lowered determination accuracy when, for example, the game apparatus 1 is used in the state where the housing 2 is maintained in a substantially horizontal direction. The determination on the mode of use may also be degraded in accuracy when the acceleration sensor 15 may possibly sense an acceleration other than the gravitational acceleration, e.g., when the user is using the game apparatus 1 while moving. The game apparatus 1 according to the present example embodiment thus determines a position of the housing 2 where the user is holding, based on the image obtained by the linear image sensor 5. The game apparatus 1 determines how the game apparatus 1 is used based on the held position. While the game apparatus 1 uses both the determination by the acceleration sensor 15 and the determination by the linear image sensor 5, a result of either one of the determinations may be prioritized. The game apparatus 1 may be so configured that the user can set which determination result is prioritized. In the present example embodiment, the game apparatus 1 prioritizes the result of determination by the linear image sensor 5. The game apparatus 1 makes a determination by the acceleration sensor 15 in the case where the mode of use cannot be determined based on the image obtained by the linear image sensor 5.

Pulse Detecting Function
The game apparatus 1 according to the present example embodiment includes a function of detecting a pulse of the user based on an image obtained by the linear image sensor 5 while the user is holding the housing 2. The game apparatus 1 emits infrared light to the outside of the housing 2 by the infrared light source 6, the infrared light being reflected by a hand of the user who is holding the housing 2 and received by the linear image sensor 5. The intensity of the reflection light of the infrared light received by the linear image sensor 5 changes in accordance with the blood flow in the blood vessels of the user. The reflection intensity of the reflection light is transferred as a pixel value in the image obtained by the linear image sensor 5.

Block diagram
fig2kuqtr.jpg



Summary
- Linear image sensors on all four sides of a handheld device
- Space Pointer Function - For example if the device is positioned on a desk, these sensors can detect location/distance of finger (sliding on desk on side of device) and move cursor/character accordingly
- Side Surface Touching Operation - When holding the device in hands, these sensors can detect touch (e.g slide finger up/down to zoom)
- Figure Detection Function - Sensors can read barcodes (e.g on Amiibos) and detect rotation/distance
- Use Mode Determining Function - Sensors can help with accuracy of acceleration sensor e.g. when rotating device to/from landscape/portrait orientation, while device is held horizontally
- Pulse Detecting Function - Detect user's pulse rate (based on the reflection/intensity of skin)
 
All of these wacky patent are starting to worry me. Gyro controls just delayed Star Fox and nobody really wants to use them anyway I don't think.
 

L Thammy

Member
Why does this handheld look so different from the other one? I guess NX handheld will come in multiple form factors,

Anyway, looking forward to the mouse game, glad to see Nintendo finally making a new IP
 

jeffers

Member
those square corners wont be kid friendly, nintendo going mature audience confirmed.

just jokes at the two button folks
 

Anony

Member
they wouldnt do a barcode type amiibo, esp if it unlocked content since it would be way to easy to duplicate that shit
 

MrPanic

Member
You guys do realize these patents are to show a new concept of a device, not to show how many buttons and control sticks are on the controller...

Got to be honest though, not liking this patent. I can't think of applications for this to appeal to me.
 
they wouldnt do a barcode type amiibo, esp if it unlocked content since it would be way to easy to duplicate that shit

I think the point of the barcode is to detect rotation/distance of the figure, in relation to the handheld.

amiibo_rotationwts5q.jpg


NFC could still be used for detection/data transfer.
 

Neoxon

Junior Member
While I could see the potential for the scrolling bumpers, I'm not sure how this would work. More so since the existing Amiibos aren't built for such image-sensing.
 

MCN

Banned
You guys do realize these patents are to show a new concept of a device, not to show how many buttons and control sticks are on the controller...

Got to be honest though, not liking this patent. I can't think of applications for this to appeal to me.

We're taking the piss out of the other patent thread.
 

Roo

Member
The vitality sensor like functionality seems like a nice way to enhance the experience depending on your pulse.
 
Top Bottom