Nintendo expanding dev tools for Wii - fur shading, predictive input, text to speech

[ziran]

Nintendo World Report - Takeshi Shimada's GDC 2007 Presentation

This looks like it was an interesting session with Takeshi Shimada from Nintendo discussing the handwriting and speech recognition tools used in Brain Training. He's directs 3 teams working on development tools and software libraries for Nintendo and third parties.

At the end Shimida discussed some new development tools they're working on for Wii:

He also briefly touched on some Wii development tools in the works that will be made available to both first and third parties. He highlighted a development tool called NintendoWare, developed by Nintendo and HAL, that emulates Wii hardware on the PC so that artists can view an accurate representation of their special effects without loading their code onto a Wii development kit. He also noted that his group is working on easy-to-incorporate fur-shading middleware and predictive input (so the game can guess what you're about to do based on prior motion). NCL is also looking into the utility of text-to-speech for Wii.

Text to speech sounds interesting, I wonder if it could be used in the Wii interface and channels, so the news, weather, etc, are spoken to you?

[Mithos Yggdrasill]

But, normally, a Dev-Kit is not supposed to have already all these things ?

[Mr. Pointy]

But, normally, a Dev-Kit is not supposed to have already all these things ?

I think the predictive speech and text to speech are pretty new-ish and awesome. Although I'm surprised about fur shading - I imagine this will be useful for when Nintendo decides to whore Starfox out to another 3rd-party developer.

[radioheadrule83]

Every graphical method employed in SMG should be added into NintendoWare.

[Cosmonaut X]

I think the predictive speech and text to speech are pretty new-ish and awesome.

Predictive input, not predictive speech. The way it's worded suggests that it's something to do with predicting the next move a player will make based on previous motions.

[cw_sasuke]

fur-shading on stafox in sfa was really impressive...

[E-phonk]

fun shading, awesome!

[CaVaYeRo]

but the NintendoWare already exists, I wonder why wasn't fur included in prior versions. Predictive input and the text thing are both interesting. The Wiimote needs a lot of work to do really awesome things.

[Nuclear Muffin]

OMG there's no Wii emulator included in the Dev kit!?

WTF, are developers even using finished dev kits? this should've been included as a feature long ago!

No wonder why 3rd party games look like turd!

[Ogni-XR21]

Speech recognition from Brain Training, hurray!

Blue

BLUE

BLUE!!!!!


BLUE BLUE BLUE!!!

[MaddenNFL64]

I think Nintendo just figured out how to simplify their TEV coding?

[Mr. Pointy]

I think Nintendo just figured out how to simplify their TEV coding?

Makes sense.

Also, doesn't AiLive come with predictive input or is that something else entirely different?

[cw_sasuke]

I think Nintendo just figured out how to simplify their TEV coding?

more good-lookin 3rd party games ftw

[Lobster]

I Like Text to Speech most..but how does that work on Wii? XD

Fur Shading..wow..

[Cosmonaut X]

I think Nintendo just figured out how to simplify their TEV coding?

Mmm. I get the impression from the various discussions I’ve read on the GC/Wii architecture, and the footage of top-end games, that the system is capable of a hell of a lot more effects-wise than nearly all third-party (and some first-party) games have shown so far. As part of the reason for that appears to be that producing these effects on the GC/Wii is done in a very different way from the other systems, anything that simplifies the process and makes those effects easier for third-parties to implement is good news.

[Chû Totoro]

He highlighted a development tool called NintendoWare, developed by Nintendo and HAL, that emulates Wii hardware on the PC so that artists can view an accurate representation of their special effects without loading their code onto a Wii development kit.

Good thing but i think it already exists ?

[Saoh]

Speech recognition from Brain Training, hurray! Blue BLUE BLUE!!!!! BLUE BLUE BLUE!!!

BLUE DAMN IT!!!!!!

[CaVaYeRo]

OMG there's no Wii emulator included in the Dev kit!? WTF, are developers even using finished dev kits? this should've been included as a feature long ago! No wonder why 3rd party games look like turd!

WTF?

I insist: the Nintyware already exists. These maybe improvements/updates.

[Stink]

isn't fur shading a bit specific?

how about bald space marine shading?

[M3d10n]

OMG there's no Wii emulator included in the Dev kit!? WTF, are developers even using finished dev kits? this should've been included as a feature long ago! No wonder why 3rd party games look like turd!

What? You think the PS2 dev kit had an emulator built-in? The only dev kits with such things are handheld ones (and the usually suck). But you're right about one thing.

Mmm. I get the impression from the various discussions I’ve read on the GC/Wii architecture, and the footage of top-end games, that the system is capable of a hell of a lot more effects-wise than nearly all third-party (and some first-party) games have shown so far. As part of the reason for that appears to be that producing these effects on the GC/Wii is done in a very different way from the other systems, anything that simplifies the process and makes those effects easier for third-parties to implement is good news.

Since TEV is GC/Wii only and there's no equivalent interface in OpenGL or DirectX, the only way to test and experiment the effects is: tweak, compile, boot devkit, look at result, rinse and repeat ad nauseum.

This is time consuming and not all developers are up to it. So a RenderMonkey-like editor which emulates all of the TEV features (can be done with one big SM3.0 shader, I think), and allows the artists themselves to set the TEV settings and see the results (and maybe even save these effects to files and load them into the game) would help a huge lot.

If you look at things this way, Nintendo had a reason for not upgrading too much beyond the GC architecture: it was the most underutilized console in the last gen. The GPU was packed with features which weren't used by most games, either due to lack of fillrate, memory, or because most devs had a focus on the PS2 (which had a very spartan GPU) and didn't even bother. So they have lots of untapped potential there and with the Wii those problems are sorted (more RAM, more fillrate, PS2 won't be the main focus forever).

All they need to do wake up the developers and get them to do some TEV-love.

[psy18]

Speech recognition from Brain Training, hurray! Blue BLUE BLUE!!!!! BLUE BLUE BLUE!!!

It's BALOO

[M3d10n]

Speech recognition from Brain Training, hurray! Blue BLUE BLUE!!!!! BLUE BLUE BLUE!!!

I wonder if it'll register if you say it in "engrish"? Anyone ever trying saying "BURUU"?

[F#A#Oo]

Fur-shading? Does this mean less bold video-game characters and more hirsute one's?

[CaVaYeRo]

If you look at things this way, Nintendo had a reason for not upgrading too much beyond the GC architecture: it was the most underutilized console in the last gen. The GPU was packed with features which weren't used by most games, either due to lack of fillrate, memory, or because most devs had a focus on the PS2 (which had a very spartan GPU) and didn't even bother. So they have lots of untapped potential there and with the Wii those problems are sorted (more RAM, more fillrate, PS2 won't be the main focus forever). All they need to do wake up the developers and get them to do some TEV-love.

that's it. TEV <3 baby!

[d[-_-]b]

Speech recognition from Brain Training, hurray! Blue BLUE BLUE!!!!! BLUE BLUE BLUE!!!

Lol and I thought I was the only one that had that problem... I'd have to say Baloo :(. Which means I've been saying blue wrong all my life?

[zallaaa]

mmm text to speech = no more possibility of voice-chat ingame?

[MachoInfinity]

Since TEV is GC/Wii only and there's no equivalent interface in OpenGL or DirectX, the only way to test and experiment the effects is: tweak, compile, boot devkit, look at result, rinse and repeat ad nauseum.

Simply not true.

If you look at things this way, Nintendo had a reason for not upgrading too much beyond the GC architecture: it was the most underutilized console in the last gen. The GPU was packed with features which weren't used by most games, either due to lack of fillrate, memory, or because most devs had a focus on the PS2 (which had a very spartan GPU) and didn't even bother. So they have lots of untapped potential there and with the Wii those problems are sorted (more RAM, more fillrate, PS2 won't be the main focus forever). All they need to do wake up the developers and get them to do some TEV-love.

There are no latent graphics features waiting to be discovered in the GameCube. Nor is the TEV something that is a mystery still waiting to be unlocked by developers. It is nothing more than a way to setup and connect multiple passes for a material. It is very straightforward and anyone who has done multi-pass rendering in OpenGL would be able to immediately get things going with the TEV.

[acidviper]

Now I can have wii fur

[_Alkaline_]

Ummm, shouldn't the wii have fur-shading already? Or it does, but it's not easy to implement? I don't get it.

Afterall Rare did an awesome job on the GC with starfox adventures.

[santouras]

Ummm, shouldn't the wii have fur-shading already? Or it does, but it's not easy to implement? I don't get it. Afterall Rare did an awesome job on the GC with starfox adventures.

they aren't talking about adding it in, they are giving devs middleware tools to incorporate these effects without having to research how to do them from scratch. Something I think all manufacturers should be doing from day 1

[rezuth]

Good thing but i think it already exists ?

No.

[MaddenNFL64]

There are no shaders on the chip, so this is needed since most devs can't be bothered to do this manually, it seems.

[Deku]

More tools for dev is always good for any console. It should bring everyone's work up a notch.

[Nightbringer]

The TEV is something similar to a Pixel Shader but it runs different and it isn´t supported in the OpenGL part of the console, you must use GX libraries for it.

This is the reason why a lot of Wii and GCN games are PS2 ports without any enhacement, OpenGL libraries on the Wii don´t support the TEV functions.

[rezuth]

There are no shaders on the chip, so this is needed since most devs can't be bothered to do this manually, it seems.

Here is the scoop on all this Wii stuff. For the first time ever this generation (Including the latest handhelds also) the least powerful console won. This confused a ton of people who were starting to wonder how this could have happened. Most of them obviously came to the conclusion it was all because of the new controller and so on put all their focus and resources into that particular piece of equipment. So because of this they think all they need to do in order to sell a game on the Wii is to take use of the new controller (Read: Waggle the shit out of the game). I would if I could encourage all new developers to take an equal amount and put it into graphics. By that I don’t mean get it as realistic as possible but a unique, clean and attractive style such as Wind Waker, Super Mario Galaxy, Okami etc…

[PantherLotus]

I support this, but it sounds like it should have been done earlier. It will be interesting to see what we get in about a year from now. Nice results should eventually show up.

[btrboyev]

OMG there's no Wii emulator included in the Dev kit!?

its pretty apparent that the only dev kits 3rd parties have had at this point are GC dev kits with early wii dev kits to make them wii games. This has been known for awhile i thought see ing all the GC dev kits with remote functionality.

[evilromero]

Nintendo should get this middleware out the door soon. If it weren't for the amusing channels and VC library I wouldn't be playing the damn thing at all.

[Squeak]

There was a patent by Nintendo posted some time ago, about text to speech input.
IMO it's much better than just a straight mic through, with maybe some vocoding like on live.

With the text you'll most likely have a sort of buffer that will let you correct sentences, and even erase them if you change your mind.
And it will let Nintendo put a filter for cuss words and deliberate misspellings.

It wouldn't really matter if it is 100% accurate, 90 - 80% accurate would be sufficient for the kind of communication going on in a game.
In the aforementioned patent, it is even suggested that the tone and loudness of the voice could be graphically represented in the letters. Like for example, angry shouting = big red letters, and silent whisper = small blue letters.
This would also allow speech synthesis to be used in games where that wouldn't sound too inappropriate (robotic voice), without the disadvantages of straight mic through and a fraction of the bandwidth use.

Speech recognition from Brain Training, hurray! Blue BLUE BLUE!!!!! BLUE BLUE BLUE!!!

Try brue.

[arcader]

Speech recognition from Brain Training, hurray! Blue BLUE BLUE!!!!! BLUE BLUE BLUE!!!

I think they use Nuance Communications (Scansoft) speech reco technology.

Its really the best in the industry and more importantly supports the most regional languages/accents.

There really is no other choice.

Or am I wrong? I know my Mario Party 6 and 7 boxes say "Scansoft" on the back - I dont own Brain Training.

[Wildstar75]

Text to speech is mostly useless in games without solid Natural language understanding and solid speech rec.

[Wildstar75]

Text to Speech is not speech recognition..

Speech to text is.

[Eteric Rice]

So this fur thing, does this mean we might see Starfox with fur again?



Hopefully, next time it doesn't look like he was thrown in a dryer. : /

[M3d10n]

There are no latent graphics features waiting to be discovered in the GameCube. Nor is the TEV something that is a mystery still waiting to be unlocked by developers. It is nothing more than a way to setup and connect multiple passes for a material. It is very straightforward and anyone who has done multi-pass rendering in OpenGL would be able to immediately get things going with the TEV.

That is the problem. Very few developer are used to complex OpenGL register combiner effects, because anything beyond diffuse+lightmap+detail required long blocks of combiner code and wouldn't run in the same way in most videocards. It also was a pain to manage if you didn't have a robust materials/Quake3-style shaders system in place (and most fixed function games didn't).

The GeForce 2 and the 4MX series were capable of dot3 bump mapping with specular (see Doom 3) and per-pixel distortion effects (pretty water) among other things, but the amount of games that used them was minimal. Most developers only touched those effects when shaders became available, and this only applies to PC and Xbox developers.

Developers coming from the PS2 didn't have anything other than vertex color+diffuse texture to work with. Most of them are used to have the game render all surfaces in the same way. So setting up inputs/outputs, indirect texture units and texture matrices to make some normal mapping is alien to them, and much harder to wrap your head around than shaders.

The TEV is something similar to a Pixel Shader but it runs different and it isn´t supported in the OpenGL part of the console, you must use GX libraries for it. This is the reason why a lot of Wii and GCN games are PS2 ports without any enhacement, OpenGL libraries on the Wii don´t support the TEV functions.

AFAIK there is no OpenGL libraries on the GCN nor the Wii. It's all GX (but it does look like OpenGL, at least for the basics).

IMO, if Nintendo or some middleware provider develops an effects system like the DirectX FX files, where you have external files which specifies all the render states and can be toggleD on and off by the game code, alongside with a WYSIWYG program for editing said files, 3rd party games would start looking much better.

[Ryuuga]

So does that include chest and back hair for Mario? Also whatever happened to banana hammock physics?

[Eteric Rice]

That is the problem. Very few developer are used to complex OpenGL register combiner effects, because anything beyond diffuse+lightmap+detail required long blocks of combiner code and wouldn't run in the same way in most videocards. It also was a pain to manage if you didn't have a robust materials/Quake3-style shaders system in place (and most fixed function games didn't). The GeForce 2 and the 4MX series were capable of dot3 bump mapping with specular (see Doom 3) and per-pixel distortion effects (pretty water) among other things, but the amount of games that used them was minimal. Most developers only touched those effects when shaders became available, and this only applies to PC and Xbox developers. Developers coming from the PS2 didn't have anything other than vertex color+diffuse texture to work with. Most of them are used to have the game render all surfaces in the same way. So setting up inputs/outputs, indirect texture units and texture matrices to make some normal mapping is alien to them, and much harder to wrap your head around than shaders. AFAIK there is no OpenGL libraries on the GCN nor the Wii. It's all GX (but it does look like OpenGL, at least for the basics). IMO, if Nintendo or some middleware provider develops an effects system like the DirectX FX files, where you have external files which specifies all the render states and can be toggle on and off by the game code, alongside with a WYSIWYG program for editing said files, 3rd party games would start looking much better.

I actually heard something about that a while ago. Someone said that Doom 3 actually ran better on the Gamecube than the X-Box because the GC was a beast with OpenGL.

Unfortunately, Doom 3 was never released on the GC due to the kiddie image. :(

[M3d10n]

I actually heard something about that a while ago. Someone said that Doom 3 actually ran better on the Gamecube than the X-Box because the GC was a beast with OpenGL. Unfortunately, Doom 3 was never released on the GC due to the kiddie image. :(

Heh? I remember Carmak bitching about how it wouldn't fit on the RAM or something. I'm glad they didn't do it, the textures look too low-res already on the Xbox version, the GC version would probably end up looking like a bumpmapped N64 game.

[Mr. Pointy]

I'm sure some crazy demosceners are porting 480p Doom 3 onto the Wii as we speak.

[Eteric Rice]

I'm sure some crazy demosceners are porting 480p Doom 3 onto the Wii as we speak.

I MIGHT buy that, as I don't have the PC version.

MAYBE... D:

[Haunted]

Nintendo improving dev tools?