Support NeoGAF

[travisbickle]

Do we really think with the amount of patches and glitches and shitty performance all round from Unity that Ubisoft coding is good enough to take advantage of the ~10% extra CPU?

Or more likely that what DF and NXGamer observed was different buggy performance in different areas on both consoles and neither assessment is wrong.

 

[benny_a]

Which is why I said you should ask Darkx10.

I should ask him because others are interested in the methodology of frame rate tests?

I've been posting in Digital Foundry threads for 5 years. Obviously I don't care much about it. I said if one cares so much about the methodology behind the gathering of data then that inquisitive mind should question it for all outlets.

Or more likely that what DF and NXGamer observed was different buggy performance in different areas on both consoles and neither assessment is wrong.

Wouldn't be the first time that Digital Foundry and other places have different results.

 

[Crossing Eden]

I should ask him because others are interested in the methodology of frame rate tests?

I've been posting in Digital Foundry threads for 5 years. Obviously I don't care much about it. I said if one cares so much about the methodology behind the gathering of data then that inquisitive mind should question it for all outlets.

You called it into question in the first place.

It's not like the DF methodology is known.

 

[benny_a]

You called it into question in the first place.

Feel free to read it as: Please link me to the methodology of Digital Foundry's frame rate analysis.

 

[_machine]

It's not like the DF methodology is known. You're just grandfathering DF into this debate.

If you aren't, then I don't know why you haven't jumped in to provide the description of the setup when I asked on the last two pages.

Yes it is, they've even posted about it on Beyond3D. I couldn't find the actual post from Leadbetter, but this should do:

DF's methodology for measuring framerate has been published many times. They measure the number of unique frames output each second (FPS or Frames Per Second) and within that measurement is a separate measurement intended to detect and quantify torn frames. Frames Per Second is how frame rates have been measures for as long as I can remember. I don't know of anybody using a different metric to FPS, or rather median FPS over whatever the capture period is.

I honestly don't get why people are so against DF lately; their facts are most of the time correct (off course they are not perfect, everyone makes mistakes), unlike NXGamer who has been proven wrong on so many of his "facts", and their conclusion is a subjective opinion of the differences. How they describe how different resolutions or effects change their experience is a subjective matter and doesn't make their facts in the analysis any less correct

 

[benny_a]

Yes it is, they've even posted about it on Beyond3D. I couldn't find the actual post from Leadbetter, but this should do:

I don't think that is enough of a description.

That they count unique frames per second seems rather self-evident when you're talking about frames per second.
If the description is going to be that reductive, there is not much of a chance that NXGamer would describe that differently.

 

[travisbickle]

Their Frame rate analysis is empirical data. It's purely based on observation, so someone else could observe something different and publish different results.

 

[Durante]

Feel free to read it as: Please link me to the methodology of Digital Foundry's frame rate analysis.

Digital Foundry uses a set of custom tools.

I think their frame-rate capturing methodology is well established, and probably the most exact thing on the internet for arbitrary sources.

(For PC, tools such as FCAT can provide even more insight)

 

[Maastricht]

I think the framerate graphs are more reliable than the numbers shown on screen, because they may vary depending on whether that uses the average of the last x seconds, the lowest in the last x seconds, etc.

And I also think it's unlikely you can even get the exact same results with the same hardware every time you measure something, let alone with different hardware with different inputs etc.

 

[Jawbreaker]

Here's what NX Gamer states in a comment left three weeks ago on a video of his.

Q: Love the frame rate captures and how accurate is it ?Does your method include frame timing ?

A: Glad you like it and thanks for the feedback. It is very accurate on the FPS but I am looking into a frame delivery method at present to bear with me on that as I work through various options. It is not a quick and easy process even for these videos.

 

[benny_a]

Digital Foundry uses a set of custom tools.

I think their frame-rate capturing methodology is well established, and probably the most exact thing on the internet for arbitrary sources.

(For PC, tools such as FCAT can provide even more insight)

Thanks. Having that custom tool be open source and see the only part you can screw up (tear analysis) would be good.

Do you know anything about them also implementing FCAT for console titles? I only know that they used it for some 970 tests.

I think the framerate graphs are more reliable than the numbers shown on screen, because they may vary depending on whether that uses the average of the last x seconds, the lowest in the last x seconds, etc.

Leadbetter has said the individual number is not to be used as a floor and their articles have generally reflected that.

 

[adelante]

It's not like the DF methodology is known. You're just grandfathering DF into this debate.

If you aren't, then I don't know why you haven't jumped in to provide the description of the setup when I asked on the last two pages.

I thought I did. Well not in direct response to your question anyway, but someone else's. I pulled up that 2009 DF article earlier : Frame Rate Analysis 101: New tools, more data, which states:

...Finding tear locations required implementation of a brand new algorithm, formulated by DF associate MazingerDUDE, following up and enhancing some interesting FPS detection work by the Japan-based ps360.

There's a huge thread over at Beyond3D where MazingerDUDE and ps360 talk at length about their methods: https://forum.beyond3d.com/threads/the-framerate-analysis-thread-part-2.44722/page-19

Their old blog seems to be around, unfortunately some of the videos (like the one here for the article New: V-Sync Screen Tear Clean-Up Tool: ) is no long available.

 

[benny_a]

I thought I did. Well not in direct response to your question anyway, but someone else's. I pulled up that 2009 DF article earlier : Frame Rate Analysis 101: New tools, more data, which states:



There's a huge thread over at Beyond3D where MazingerDUDE and ps360 talk at length about their methods: https://forum.beyond3d.com/threads/the-framerate-analysis-thread-part-2.44722/page-19

Their old blog seems to be around, unfortunately some of the videos (like the one here for the article New: V-Sync Screen Tear Clean-Up Tool: ) is no long available.

Based on that DF article I assume they use the percentage approach to tearing which means an unique teared frame can actually count as something like 0.8 frame.
That part wouldn't be applicable here because AFAIK the game doesn't tear on consoles.

 

[yurinka]

DF did their performance analysis of the game with the day 1 patch installed.

Yes, suspiciously 'forgot' to mention that PS4 pre-patched was unlocked and sometimes got up to 45 fps.

Another reason more to think they are clearly biased and try to hide or to show only a portion of the information to make Xbox look better against PS.

Another reason to think we have to skip DF and find other analysis sources.

 

[Crossing Eden]

Yes, suspiciously 'forgot' to mention that PS4 pre-patched was unlocked and sometimes got up to 45 fps.

Another reason more to think they are clearly biased and try to hide or show only a portion of the information to make Xbox look better agains PS.

Another reason to think we have to skip DF and find other analysis sources.

Is it really that hard to believe that they only started testing the framerate after the d1 patch was installed and thus never knew about the unlocked framerate of the unpatched ps4 version?

 

[SniperHunter]

Yes, suspiciously 'forgot' to mention that PS4 pre-patched was unlocked and sometimes got up to 45 fps.

Another reason more to think they are clearly biased and try to hide or show only a portion of the information to make Xbox look better agains PS.

Another reason to think we have to skip DF and find other analysis sources.

Lol

When i played unreal 2 on my shitty hp machine that had one of those intel integrated gpus. I played through the entire game at 10 fps. The frame rate only went up when i was looking at the sky

 

[Javin98]

Hmmm an ND designer retweeted NXGamer's video. Obviously that does not mean any kind of endorsement on the technical merits of the video but could be as well...

https://twitter.com/thegreatbluebit/status/541887192646180865

Anyways there has to be an objective interpretation to framerate issues and if these frame rate counts are right DF will have to be more closely scrutinized I would think.

Was he definitely wrong on bits of his analysis of UC4 though?

Someone from naughty dog posted his analysis....the plot thickens....

https://mobile.twitter.com/thegreatbluebit/status/541887192646180865


Haha above post... how spookys that.....

Could it be?! Tessellated rocks for Uncharted 4 confirmed!

On a more serious note, it could be just appreciation from ND for the well detailed albeit slightly inaccurate tech analysis.

On-topic, I occasionally enjoy watching NXGamer's tech analysis more than reading DF's articles. I'm with those who find DF's articles slightly biased from the start. How is it that they conveniently missed out the missing foliage on the XBO version of GTA 5 until it became known to the whole internet? They didn't have this problem in Red Dead Redemption when the 360 had the advantage. And let's not forget the removed god rays in Far Cry 4 and uncapped frame rate on AC:Unity on PS4 without patches. Regardless, DF is still by far my main source for technical analysis but I wish NXGamer could be a worthy competitor or any other site for that matter

 

[Windom Earle]

How is their framerate analysis method crap? Don't they use a program that accurately measures framerates? I find the the recent surge of contempt for digital foundry very strange.

Because they don't compare avg framerates, instead they often seem to rely on randomness (See GTA5).

Why not run through a scenario five times on each console and then compare the average?

I think this would have avoided a lot of controversy with their recent articles.

edit: also you must be new here... "recent" contempt, lol.

 

[eso76]

Even if NXGamer isn't entirely accurate it's good to have another person/site out there doing this level of analysis for some fresh perspective.

These guys are counting frames, it shouldn't be a matter of perspective !

Shouldn't.

 

[ypo]

Thanks. Having that custom tool be open source and see the only part you can screw up (tear analysis) would be good.

Do you know anything about them also implementing FCAT for console titles? I only know that they used it for some 970 tests.


Leadbetter has said the individual number is not to be used as a floor and their articles have generally reflected that.

He's bullshiting. They always used the numbers even when they don't match the real time graph, which strangely only happens to PS version of games.

 

[KKRT00]

He's bullshiting. They always used the numbers even when they don't match the real time graph, which strangely only happens to PS version of games.

And conspiracy theory still keeps going.

Why do i even read this thread anymore? ...

 

[ypo]

Lol so don't read it and stay out or are you pulling a leadletter? I know facts are hard for you to swallow sometimes. You can just stay with hur hur just coincidence hur excuse though.

 

[Dictator93]

Lol

When i played unreal 2 on my shitty hp machine that had one of those intel integrated gpus. I played through the entire game at 10 fps. The frame rate only went up when i was looking at the sky

lol

Also, anyone find out how the framerate is measured yet?
-----------------
COnspiracies about DF having a bias towards x or y is pretty funny btw.

 

[RoboPlato]

Yes, suspiciously 'forgot' to mention that PS4 pre-patched was unlocked and sometimes got up to 45 fps.

Another reason more to think they are clearly biased and try to hide or to show only a portion of the information to make Xbox look better against PS.

Another reason to think we have to skip DF and find other analysis sources.

They only do FPS analysis after the day 1 patch is installed.

 

[kitch9]

Yes it is, they've even posted about it on Beyond3D. I couldn't find the actual post from Leadbetter, but this should do:


I honestly don't get why people are so against DF lately; their facts are most of the time correct (off course they are not perfect, everyone makes mistakes), unlike NXGamer who has been proven wrong on so many of his "facts", and their conclusion is a subjective opinion of the differences. How they describe how different resolutions or effects change their experience is a subjective matter and doesn't make their facts in the analysis any less correct

It is because they appear to have gone from just reporting the performance facts to reporting vague apologetic and not quite frank descriptions of "gameplay" since the new consoles have launched.

Stuff that was important and definitive no longer is, apparently they can no longer see scaling artifacts but used to be able to spot them a mile away and they are willing to ignore that one version has 30-40% less pixels if another version drops a few frames when you throw a grenade which obscures your vision completely anyway.

Saying that it is a very subtle shift in narrative, but Leadbitter is a self confessed MS fanboy and MS look to buy their largest ad spaces so it's understandable... We'd all to the same.

We need guys like Nxgamer and others to keep DF on the straight and narrow. DF over the years have become the defacto performance analysis site and them alone. Probably not the best scenario we should have.

 

[DieH@rd]

He just uploaded new video

Assassins Creed Unity PC Final Verdict: AMD,Nvidia and Intel compared
http://www.youtube.com/watch?v=GdsycPtUyyY