• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
  • The Politics forum has been nuked. Please do not bring political discussion to the rest of the site, or you will be removed. Thanks.

Hype Ninty files patent for DLSS 2.0 proprietary 4K technology

Marty-McFly

Banned
Jul 2, 2021
1,712
4,250
465
The 4K future for Switch Advance looks inevitable (BS rumors from Bloomberg aside).

Interesting Nintendo is choosing to go with their own DLSS solution.


What year this comes out who knows. I'm still definitely all over OLED because who knows when we'll actually see this?

Will it be an extension of the current Switch or is this Switch 2?

and how will this effect its surging third party support?

patents for "SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION". AI upscaling akin to Nvidia's DLSS and Intel's XeSS.

patents were filed on March 25, 2020 and published today. the inventors are all listed from France, patents mention the methods can be accelerated by matrix multiplication hardware, like Nvidia's Tensor Cores



FAjrNlBX0AULp3W



patents are viewable here

SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION - NINTENDO CO., LTD.

A computer system is provided for converting images through use of a trained neural network. A source image is divided into blocks and context data is added to each pixel block. The context blocks are
www.freepatentsonline.com
www.freepatentsonline.com

SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION - NINTENDO CO., LTD.

A computer system is provided for converting images through use of a trained neural network. A source image is divided into blocks and context data is added to each pixel block. The context blocks are
www.freepatentsonline.com
www.freepatentsonline.com

 
Last edited:

martino

Member
Apr 25, 2013
6,901
4,905
810
France
Dunno lot of uncertainties at the moment...
Will games push more cpu for something not graphics related at some point ?
Will ml or shader usage explode for multiple game usages not only graphics related too ?
If games pushes one of the two dlss will not some anything for them for a 10 to 20 w apu.
 
Last edited:

MrSec84

Member
Jul 1, 2020
112
151
230
Switch 2 for sure, this will be years away though.
I'm doubtful of this, given that even though the Wii was a crazy success for Nintendo they still launched the Wii U 6 years after that platform, the tech on the Switch is well behind the PS4 or XB1, let alone the PS5 and XBox Series consoles. Nintendo now has a taste for the 3rd party sales it has gotten on Switch, seems like they'll release a new platform with a similar hybrid style to Switch late next year or sometime in 2023.

Valve has shown that the hardware exists to be able to at least run a baseline level of hardware capable of running any current multiplats and that's using hardware that doesn't have DLSS level capabilities yet, NVidia just has the tech already.
Best thing is that DLSS can be used on low power mobile devices, the performance mode of DLSS 2.0 or higher works great and there's no reason why Nintendo can't have developers coding those features into their games from day one of game development.

By this time next year Tensor hardware is going to be incredibly mature, same goes for Ray Tracing, chip shortages should also be much improved, we likely would have had at least a pro version of the Switch this year without the manufacturing issues that have plagued the world during the Pandemic.
 
  • Like
Reactions: NeoIkaruGAF

Barakov

Member
Sep 30, 2006
9,171
9,216
1,670
The 4K future for Switch Advance looks inevitable (BS rumors from Bloomberg aside).

Interesting Nintendo is choosing to go with their own DLSS solution.


What year this comes out who knows. I'm still definitely all over OLED because who knows when we'll actually see this?

Will it be an extension of the current Switch or is this Switch 2?

and how will this effect its surging third party support?



 

GametimeUK

Member
Jul 2, 2015
2,053
1,859
620
Part of me wish they release a switch 2 in 2022, that would create a lot of drama among the oled buyers xD
It wouldn't be the first time Nintendo released an updated console and then a successor the following year if I remember correctly. I believe the DSi XL released in 2010 (outside of Japan) and the 3DS released in 2011. Unless someone can correct me or provide another example of when they've done this.
 

Marvel14

Member
Sep 30, 2020
510
737
445
Nintendo creating their own DLSS is like a monkey designing a space ship.

Why would they unless they drop nvidia?

My thoughts exactly. Is this their way of letting their customers know it's coming?

Haven't seen anyone mention of this but Nintendo now seems to bring out their new box a couple of years after MS/PS 's latest. This ensures no rapid copying by its competitors (as with motion control) and a cheaper, higher power offering than they would otherwise be able to offer, that balances cost and price point. So 2022-2023 is definitely the time for their next console.


I don't see them having any expertise at all with DLSS and they lose all the benefits of a wider industry standard and all the upgrade benefits that would bring.....

This is either more fresh bullshit or a comms stunt.
 
Last edited:

Coolwhhip

Gold Member
Aug 26, 2019
4,924
15,419
670
They are still likely working with Nvidia.

Saying that godfathers of gaming couldn't create well designed high tech hardware if they wanted to is ridiculous.

You must have missed the Gamecube and N64.

It doesn't make sense for someone like Nintendo to try and compete with Nvidia and AMD on something they are an expert on by now. But eh, patents don't mean much anyway.
 
  • Like
Reactions: NeoIkaruGAF

Marty-McFly

Banned
Jul 2, 2021
1,712
4,250
465
It doesn't make sense for someone like Nintendo to try and compete with Nvidia and AMD on something they are an expert on by now. But eh, patents don't mean much anyway.
They could very likely still be working with Nvidia but putting their own spin on the tech (in fact, I think they are partnered contractually for the next decade). Let's not forget that the off the shelf Tegra Nvidia provided for the Switch was hacked in a matter of weeks. I'm sure Nintendo wants to take a more active approach in the design process especially considering how successful the Switch was conceptually. Refining the tech is now of much more importance than selling a new concept.
 
Last edited:

Marvel14

Member
Sep 30, 2020
510
737
445
They are still likely working with Nvidia.

Saying that godfathers of gaming couldn't create well designed high tech hardware if they wanted to is ridiculous.

You must have missed the Gamecube and N64.
You really are a Nintendo spokesperson aren't you?

Any company that hasn't been working at the cutting edge of complex electronics manufacture for decades would not be good enough at to go it alone.

Especially when their competitors have been pushing the envelope during all that time.

Nintendo have worked with IBM, Qualcomm and many other high tech companies to provide bespoke tech into their products for decades.

Saying Nintendo could do all of that by itself is what's really ridiculous. And only someone advertising for the Nintendo brand would be blinkered enough to say it.
 
Last edited:

Marty-McFly

Banned
Jul 2, 2021
1,712
4,250
465
You really are a Nintendo spokesperson aren't you?

Any company that hasn't been working at the cutting edge of complex electronics manufacture for decades would not be good enough at to go it alone.

Especially when their competitors have been pushing the envelope during all that time.

Nintendo have worked with IBM, Qualcomm and many other high tech companies to provide bespoke tech into their products for decades.

Saying Nintendo could do all of that by itself is what's really ridiculous. And only someone advertising for the Nintendo brand would say it.
Who said they are going at it alone? And the presumption that the most successful and established gaming company in history doesn't have the resources to be "good enough" to design their own tech is laugh inducing. No offence.
 

Dream-Knife

Member
Feb 22, 2021
2,170
2,494
430
Those drawings show upresing a 540p image to 1080p (unless I'm misunderstanding). So, I don't think this is what people think it is.
 

Marty-McFly

Banned
Jul 2, 2021
1,712
4,250
465
Those drawings show upresing a 540p image to 1080p (unless I'm misunderstanding). So, I don't think this is what people think it is.
Definitely misunderstanding. This is AI upscaling for those savvy enough to read through the proposed patents. Mentioning Tensor Cores is pretty much confirmation this is some DLSS 2.0 variant or something
 
Last edited:
  • Like
Reactions: Skyfox

Marty-McFly

Banned
Jul 2, 2021
1,712
4,250
465
Multiple possibilities. Note that we've known for a while that NERD actually does have their own machine learning engine, which was used to upscale the pre-rendered videos in Super Mario Sunshine.

1 Nintendo wants to have their own solution for future proofing, so that if they ever do split from Nvidia they will still be able to use ML upscaling
2 Nintendo built their own ML solution that plays better with their own internal engines
3 Nintendo built their own ML solution that plays better with older legacy content
4 Nintendo built their own ML solution that plays better with their graphical style than general DLSS does

The patents reference using Nvidia's tensor cores to implement the ML software solution, so it does seem like the infrastructure used will be Nvidia's, but they obviously want full control over the algorithm.
 
Last edited:

lils

Member
Sep 1, 2021
158
227
235
it seems strange that they would develop their own when nvidia and amd have poured billions into this
 

StateofMajora

Member
Aug 7, 2020
2,240
2,919
385
Neat. Although it's probably dlss, maybe Nintendo are working with Nvidia to improve it for their custom tegra chip. It's worth pointing out that whatever kind of chip the switch 2 uses, it will need to be more efficient at machine learned image reconstruction due to inevitably far fewer tensor cores than even an rtx 2060.

Nintendo also may not like aspects of current dlss like sharpening or ghosting, or how it can occlude small particles i.e. Death stranding.

I just hope this thing comes out before metroid prime. I wanted bay 3 to be on switch 2 but I lost there lol.
 

Marty-McFly

Banned
Jul 2, 2021
1,712
4,250
465
it seems strange that they would develop their own when nvidia and amd have poured billions into this
As I mentioned the patents reference using Nvidia's tensor cores to implement the ML software solution, so it does seem like the infrastructure used will be Nvidia's, but they obviously want full control over the algorithm.
 
Last edited:

Kazza

Member
Oct 6, 2018
4,062
9,241
750
Nintendo creating their own DLSS is like a monkey designing a space ship.

Why would they unless they drop nvidia?

I kinda agree with this. If creating your own version of DLSS is so easy, why have AMD been struggling with it so much? (and it goes without saying that they have much more experience in GPU design than Nintendo)

It would be a typical Nintendo dick move to try and stab Nvidia in the back though :messenger_tears_of_joy:
 

MagnesG

Banned
Apr 20, 2019
3,099
2,492
515
Only Nintendo could touch their own content and releases. This is likely for internal implementation with their future games developmen.
 
  • Like
Reactions: FloatingIslands

Nautilus

Member
Feb 3, 2021
372
612
300
This is obviously tech for Nintendo's 10th gen, assuming Nintendo actually go through with it.

Hard to say when thry will launch it, but if I had to guess, the earliest I can see it happening is 2024. If its earlier or later than that, I believe it will mostly depend on the Switch performance in 2022.
 
  • LOL
Reactions: MrSec84

Soodanim

Member
Feb 24, 2012
8,388
4,194
1,130
United Kingdom
It makes sense for Nintendo more than anyone else because they use modest hardware. The benefit will be greatest on their consoles. Nothing but a good thing. How good can a native 1080p image look with upscaling tech these days?
 

Woopah

Member
Jun 15, 2019
2,250
2,520
410
It's definitely technology that can help Nintendo a lot. Upscaling could help with both battery life and third party support so will be interesting to see the results.

Question now is whether they position the next model as a revision or a successor.
 

Stitch

Member
Nov 26, 2006
10,630
4,131
1,785

Kenpachii

Member
Mar 23, 2018
9,922
12,302
815
Could be they want to have it enabled on all devices even the current switch which nvidia doesn't wanna support with DLSS.
Could also be they going to ditch nvidia next round.
 
  • Thoughtful
Reactions: borborygmus

Rikkori

Member
May 9, 2020
2,704
5,131
470
That's very smart, because it allows them to maintain the technology required for the games in case they want to switch SOC vendors (i.e. AMD/InteL) in the future. Really the most expensive part for developing something like DLSS will be the hardware-time to do the training, but it's not a big deal at all. People way overrate how advanced something like DLSS is or what it does just because they hear "AI" and let their imagination run wild.
 

jigglet

Member
May 18, 2020
3,614
6,544
630
Ninty: we patented this tech
Industry: oh, we moved on 17 years ago
Dennis Dyack: for or against?
 
Last edited:

KungFucius

Member
Jul 16, 2008
2,314
1,256
1,280
what happened to gameplay over graphics?
Isn't this really gameplay over graphics? I get that you are just trolling, but you missed the mark just like most trolls.

Develop low res game with good gameplay, upscale to higher res with hardware that is cheaper than the hardware to render without this algorithm. This prioritizes gameplay over graphics and cost of both development and HW.
 

yurinka

Member
Jan 19, 2007
14,809
7,072
1,895
Barcelona, Spain
www.capcom-town.es
I gave a quick look at the patent and sounds like DLSS: cloud based, machine learning based image reconstruction to upscale resolution depending on the client's resolution (720p, 1080p, 4K, etc).

I don't know the details of Nvidia's DLSS, but I know it requires a specific GPU hardware and game specific implementation, which makes me think Nintendo would apply this in a totally different hardware where all games implement it (Switch's next gen successor, call it Switch 2 or Super Switch). What is the difference between this an Nvidia's solution? If it's the same, why Nintendo patents it instead of Nvidia? Didn't Nvidia already had it patented?

4k on a mobile size display is stupid.
No, there are smartphones with 4K displays and they look great. Over time the resolution of phones and tablets increase and their price decrease. It's just a matter of time that they will become standard.

Those drawings show upresing a 540p image to 1080p (unless I'm misunderstanding). So, I don't think this is what people think it is.
Yes, but in another place it mentions other resolutions like 4K.


As I remember they said they expect Switch to last around 4 years, so I assume they'll release its successor 1 or 2 years before that. I'd say 2024 is more likely.
 
Last edited:
  • Like
Reactions: Guilhermegrg

Buggy Loop

Member
Jun 9, 2004
6,773
4,216
1,735
Quebec, canada
I really hope they let Nvidia handle it and that it’s just they want a proprietary solution like their API, but developed by Nvidia.

I can’t imagine Nintendo not fucking it up somehow, with terrible latencies or poor image quality if they were to attempt this.
 

jigglet

Member
May 18, 2020
3,614
6,544
630
I really hope they let Nvidia handle it and that it’s just they want a proprietary solution like their API, but developed by Nvidia.

I can’t imagine Nintendo not fucking it up somehow, with terrible latencies or poor image quality if they were to attempt this.

Agreed, why reinvent this wheel, when your reinvention is only going to be worse.
 

Ev1L AuRoN

Member
Oct 30, 2020
489
768
370
540p should reconstruct back to 1080p like Nvidia's DLSS balance. For a 4k output the base resolution must be at least 1080p native.
 

StateofMajora

Member
Aug 7, 2020
2,240
2,919
385
540p should reconstruct back to 1080p like Nvidia's DLSS balance. For a 4k output the base resolution must be at least 1080p native.
Ultra performance dlss is 720p to 4k.

But if Nintendo were to use dlss on their own software it probably wouldn't start so low, but it could vary from team to team.

This gen there's a wide gap in resolution Nintendo uses, i.e. Yoshi crafted world with UE4 and nice GI lighting is 540p while mk8 is 1080p so yeah lol.
 
Last edited:
Feb 20, 2021
501
464
260
Anything that tries to be 4K, but not natively 4K, looks artificial to me. Nvidia's AI upscaling looks really bad (identical to cranking the sharpness up), and I don't like the artifacts (stippling, for lack of a better word) created by checkerboarding.

Not too hyped on this tech, but I'll take anything at this point over a Switch. Games are starting to look gross outside off Nintendo's stuff.
 
Last edited:

Kenpachii

Member
Mar 23, 2018
9,922
12,302
815
In control DLSS 2.0

4k resolution u got 4 options

1) quality = 2560x1440
2) balance = 2227x1253
3) performance = 1920x1080
4) ultra performance = 1280x720