DXO Lenses for D800

PB_PM

Well, I think haroldp is correct. I do suspect we will see a significant sharpness difference between the 24 mp no AA filer D7100 and the 16 mp D7000 AA filter when the same subject is shot with the same lens. These sensor only tests of color depth, dynamic range and ISO just aren't measuring the added resolution of the D7100 sensor when coupled with very sharp lenses.

You might notice the difference at 100%, but in standard print sizes? Meh.

TaoTeJared

How about the D7100 scoring the same as the D7000 when it clearly should be better from the images I have seen? Does that prove DxOMark is wrong or does it prove my eyes are mistaken?

Are my eyes mistaken? Is the ardent supporter of DxO Mr. Donaldjose starting to see the light?

From the sample images I have seen so far, I think the D7100 IQ is better and much better than what the tests state at high ISOs. It has clearly better options/functions, (AF, Metering, Video) on it for sure.

donaldejose

PB_PM: Yes, I don't think you will see a difference at 8x10 print size. In fact, I doubt you can see a difference between FX and the most recent generation of DX sensors at 8x10 print size. Much of the detail in the images captured by my D800 are not visible at 8x10 or on my 23 inch computer monitor. As KR says with his usual degree of exaggeration: "The D7100 is new in that it has an ordinary 24 MP sensor, but without an anti-alias filter for added sharpness. Honestly, it won't make much, if any, visible difference at less than 6-foot (2-meter) wide print sizes, and only if you use the very finest NIKKOR lenses and technique."

TTJ: I suspect DxOMark does accurately measure whatever it is they measure but I also suspect there are many components to IQ which their testing does not measure. I will be interested to see the two scores they give to the 85mm 1.8 G on the D5200 and on the D7100 and compare it to the score of the same lens on the D800.

TaoTeJared

I will be interested to see the two scores they give to the 85mm 1.8 G on the D5200 and on the D7100 and compare it to the score of the same lens on the D800.

That is real easy - just take the D3x score and add a point or two and you will get it. Same size sensor, and the "sharpness" will increase by 1-2 due to the lack/modified of AA filter. It is what all of there tests, between bodies do.
My prediction
7100 = 37
5200 = 36
Actual scores:
D800 = 40
D3x = 35

This is my point with their "tests" - If I can do it in my head in a split second - why do you need a test or better, what good is it?

heartyfisher

@TTJ : I agree that we dont exactly know how they derive their numbers but after looking at the PMPix of several lense and camera combinations I too at first was confused.. But after doing some calculation and correlating between cameras and lense combinations i find that it is consistent and therefore I am confident now that whatever PMpix number they bring up for a lens vs lens on the same body . the relative quality of the image will be correct.. ie if for the same camera one lens comes out with a 6 PMPix and another with 7 then I am confident that the image quality of the second lens is better than the first.

Just as an aside if a lense shows 6 Pmpix on a dx camera you would expect all things being equal that it would have 6 * 2.2 == 13 PMpix on an FX camera. if it is not then you can surmise that either the DX sensor is more efficient than the FX sensor or the other way around. after looking at several combinations it was clear to me that sensor efficiency has improved over time which is to be expected :-).

One curious thing I found was that the newer sensors are less efficient than expected.. which leads me to conclude that there has been some effect on efficiency from the high MP. .. this makes sense as we all expected that the High MP would result in noise which we do not see. From the Dxo results I conclude that this "noise" has been masked but is revealed in the lower than expected PMpix.

If the Dxo data is crap .. it wouldn't show this. So how ever it is they derive these PMpix values they have been applying it consistently. And I think there is value in that. Furthermore all other data that I have seen on their site seems consistent and provided in good faith. So although I dont take much stock in the final number which is derived from some weighting system that would probably be different from My weighting, I am confident that the raw data is accurate to show relative differences between the items being tested.

Paperman

Left between 2 choices - an opinion from a user and the other being a lens MTF test - in deciding if a lens is sharp, I would always go for the scientific test result. The human eye can not see 50 lines in a mm.

There is no other site where you can find tons of lenses tested on tons of bodies. Photozone was OK 5 years ago but now it is so dated ; a lens tested on a 8-12 Mp camera 5 years ago is not tested again on any new cameras with 16-24 Mp

I don't know why but it is always those pros with expensive gear & glass who seem to be unhappy with the DxO results. Maybe they are right - because they have tested all that equipment and they know . But is it also possible that they are just blinded by the fortune they have paid for their lenses and they don't want to see them perform worse than a $300 lens ?

Can anyone point out specific examples why they think DxO results are inconsistent/ not reliable ( other than saying " I have used those lenses and they are not what it says in tests" ) ? Like another test site which has totally different results for the same lens. I am not trying to start an argument , I just really would like to know why I shouldn't rely on some numbers/ test results when there is nothing else ? To be honest, I do think the criteria they use for "general score" is not what it should be ( sharpness should be the main one / carry more emphasis ). But still it gives an opinion and there are specific figures if one wants to look at that particular aspect.

TaoTeJared

@heartyfisher - I am not confused at all. I use to design metrics for multiple fortune 1000 companies for 5 years before doing photography. I have seen dang near every single way people roll numbers up, calculate them and then market arbitrary numbers as something real. I can back into most of the roll-ups but the original "base" numbers I can not ascertain what they come from. Hell, they even say they made it up. Your statement about the scores showing new sensors are losing efficiency is one of the main things I have noticed. And No it does not make sense at all if you looks at sensor manufacture tests that show they are actually gaining sensitivity - higher iso is one example, deeper color saturation/dynamic range is another, but then the marketing comes in and tries to explain why their tests don't show this. Applying garbage consistently only creates a dump, not a mecca of goodness.

@Paperman - myself and a few others have pointed to things in multiple threads that you can go back too.
One of the main issues is that their tests can not be directly compared to other's test software, MTF etc. They have decided after 100 years of optical bench tests and proven techniques, to make up their own that are loosely based on standards but tweaked for their end use of software designing. Mostly what it seems is they have made tests that are cheaper to do rather than better.

Photozone's site is quite outdated on many tests and they don't do many updates at all. I have seen some updates with a D3x though. The D200 tests need to be updated badly though. Afterall it is just 3 photographers who do it in their free time and not a company that makes money from their work.

Look up "Imatest lens tests" and you will find companies/people testing various lenses. Unfortunately there isn't too many people/companies who just spend $100,000 a year to acquire lenses, bodies and just sit and test the stuff. As we all know it is not a cheap endeavor. I have seen some photographers/testers that actually sell their lens tests - aggravates me and I have never paid for them, but it does cost them money to do it. The few I have read are very well done and use the Imatest software.

Paperman

The reason I am asking for specific examples TTJ is because I don't remember any. I know it was discussed various times but it was always an "opinion" from someone . Like us now saying the 7100 should be better than the D7000 or vice versa ... Are there any test results contradicting any of DxOs findings - that is what I want to know.

I remember a discussion in a forum about the DxO rating of the new Canon 70-200mm f2,8 . The fact that DxO said the older version was sharper drove owners of that lens crazy - each insisted on how sharp their lens was and that DxO was fiddling with the results. The outrage was such that DxO had to add an explanation to their findings . They said their conclusion was based on the asymmetry found in the sharpness of the right and left sides of the frame - one side being visibly softer at close distances. The finding was also backed by DPreview ( 3 years ago - way before they started cooperating with Dxo ).

roombarobot

Do folks also not trust the DXOMark sensor results? I don't hear as much about those as I have about the lens results, particularly since they changed their lens scoring. Well, maybe I'd hear more complaining if I was on a Canon board.

So, do people not like their sensor tests? Are there good alternatives?

Paperman

Just saw and read the last pages of the other D800 topic where the subject of DxO sort of exhausted everyone ; wouldn't have mentioned it here if I saw that one first.

Still, every time asked why the DxO results are not credible and should not be trusted, the discussion seemed to have shifted to "photography not being about science " . Of course, it is not only about science and of course it is not only the sharpness/color depth/tonal range that counts when it comes to rating a lens. But what harm is it knowing those on the side ? Would you still buy a lens if the feel & bokeh etc is good but it is crap in sharpness ?

heartyfisher

"Would you still buy a lens if the feel & bokeh etc is good but it is crap in sharpness ?" of course the answer to that is yes! and people do pay a premium for those qualities ! but like you said .. what ever metric you can measure should be measured.

sevencrossing

just looked at Dox on the new vs old 80-400
DOX complexly misses one of the big improvements of the new lens - speed of focus

( hope this is in the right place I did a forum search but most of the results are for the old forum )

FlowtographyBerlin

So remember, DxO does NOT tell you which are the best lenses... they tell you which have the best OPTICS.

Very well said.

TaoTeJared

So remember, DxO does NOT tell you which are the best lenses... they tell you which have the best OPTICS.

Actually they do not - and not even close to it. This is the largest misconception of their non-repeatable results that are produced behind a cloud of "metric" smoke.

What they really do is to try not to make companies angry at them and make users feel good about what they own. Oh and they measure distortion and light transmission well - but so does everyone else. They are ok at software design, good at writing without sounding like engineers, and their true specialty, is at marketing themselves.

Paperman

What they really do is to try not to make companies angry at them and make users feel good about what they own ....

Well, Canon guys hate DxO to their guts :x

Msmoto

Interesting that I cannot find on DxO, the super telephoto Nikkors... lots of Canons, but no Nikkors. The issue of DxO is that the lens is tested.

Also, they have a list which suggests for a 16" x 20" print one needs a DxO score of 40....wow, I guess my shots are all not very sharp, then......the 85mm f/1.4 and f/1.8 Nikkors are the only ones with 40.....

And, so the confusion continues. However in looking at various "tests" by DxO I do believe one can use the information as a piece in the decision making puzzle when one is interested in purchasing a new lens. Rather like automotive road tests....the conclusions do have a lot of bias based upon multiple factors.

Thus, the only way I can determine the best lens for my camera is to purchase it from a reputable dealer, test it out and if it does not perform....send it back.

TaoTeJared

It's not fair to say "they don't measure optics." I think your gripe is with the disclosure of their methods and how they arrive at a single number to measure the lens' overall quality. Possibly they've given a bad review to a lens you actually like. Fair enough, but I'm sure their results are repeatable in theory, they just don't provide you with the means to do it. In the same way, however, Canon and Nikon don't exactly provide detailed white papers on how, exactly, they obtain their MTF results.

PitchBlack - the longer you stick around this forum, you will find many of us do not trust DxO's scores and think they are nothing more than just a marketing tool to sell software. We have gone over them many, many times any what they lack.

1) You can repeat Canon and Nikon's MTF charts any day if you have the lab - they are ISO9000 standards based. The reason why it is hard to do the tests is that they have the best equipment in the world to test their lenses with. Canon, Nikon, Zeiss, Leica, Olympus, all make lenses that are used in high end scientific equipment, space programs, government weapons systems, and almost anything else you can think of. If you search on Zeiss's site they actually have videos that show how they test their lenses by the standards. They use lasers, high resolution test charts printed on glass, and a lab without limits on how much to spend on equipment. It's absurd to think that publication's that are always close to bankruptcy or a sub $1000 software is better.

2) DXo Scores their lenses on a metric system (assigned numerical values for results) which they never explain their rational or how they assign scores. That lack of transparency is why most are skeptical. If you read their "how they do it" they clearly state they do this and they will not release what the rational is. The end result is that scores of every lens are so numerically close, that it is insignificant. Owning many lenses currently (and many others that I have swapped out) that are in their scores, I can say there is a much larger difference than what they try to state. Some are better, some are worse, very few are that close in some sort of score.

The only tests I find to be accurate is their distortion and light fall off. Everything else I take with a grain of salt.

It is simple, they need to sell software to stay alive. If they make people angry by showing "their" lenses are not as good as they thought, or worse, a sub $1000 lens is better, people are frustrated with them. Frustrated people don't buy software from the source of that frustration.

True MTF and accurate lens scores do not change with each body or sensor format, MP count. The idea that those somehow change with different sensors is a very new phenomenon that people have been either tricked into believing that is the norm, or have not taken the time to inform themselves of the pre-digital era.

TTJ: I suspect DxOMark does accurately measure whatever it is they measure but I also suspect there are many components to IQ which their testing does not measure. I will be interested to see the two scores they give to the 85mm 1.8 G on the D5200 and on the D7100 and compare it to the score of the same lens on the D800.

What is curious is that rarely do you see the same lens on two different bodies with the same sensor size (mp count). Scores different? I could go on for pages on all the never ending idiocentricties that make up their database.

FlowtographyBerlin

2) DXo Scores their lenses on a metric system (assigned numerical values for results) which they never explain their rational or how they assign scores.

Which is why I find it very strange how much impact these actual scores have. Which in turn underlines your point how much this is a marketing tool.