DxO Mark Ratings - are they any use at all?

spraynpray

I'm sure there are quite a few people on this forum that understand DxO Marks better than me, but my question is are they serving a purpose as-is? I just looked at the 24-120 and 24-70 f2.8 on the D7100 and D610 and came away only knowing that the D610 scores quite a bit higher (whatever that means) even though it is worse for CA, worse for distortion and worse for vignetting but better for sharpness. I just time-out when reading the tests (some form of geek fatigue or A.D.D.).

I wonder if there is some kind of conclusion they could display (maybe a standard image @100%?) that would give readers more of a real-world idea of what changing lens 'A' for lens 'B' would give them.

I'm sure the answer isn't easy, but it must be possible, surely?

WestEndBoy

I know many people will think that I am swearing but Ken Rockwell's site is often quite useful for this.

spraynpray

I don't think of KR as knowing much about the tech side of things - he knows how to make a living off what he does, but that's about it. Maybe I'm wrong, but I've seen him change his views depending on who's paying him so he has no cred with me.

Vipmediastar_JZ

I use DxO for comparisons but then do a search on Flickr for actual images to get a better view.

As for KR when im looking for some stuff on google his site has the answer.
For example I needed some info on a 200mm f4 ai lens and a quick search his site came up with the answer.

I can't say his stuff is useless or usefull but he does have the right info when I need it and so does other sites like MIR

spraynpray

I just checked the 60mm G, the 60mm D and the 105 G on a D7100 and they are all the same. Nikon are wasting their time making new lenses according to DxO!

manhattanboy

I'm sure there are quite a few people on this forum that understand DxO Marks better than me, but my question is are they serving a purpose as-is? I just looked at the 24-120 and 24-70 f2.8 on the D7100 and D610 and came away only knowing that the D610 scores quite a bit higher (whatever that means) even though it is worse for CA, worse for distortion and worse for vignetting but better for sharpness.

The tests are accurate. The 610 has a much larger sensor than the 7100, but both have the nearly same megapixels. There is more "slop" allowed in how the lens focuses onto each pixel in the 610 versus the 7100. The result is a "sharper" image, albeit one with a very different point-of-view. The CA, distortion and vignetting are all worse on the 610 because it is making use of the outer portions of the lens, whereas the 7100 is using the central "sweet spot". Overall DXO is very useful, we just might not like what they say :P

Personally I have found DXO to be on the money for the 7100. It said that the 70-300 is not sharp enough for that camera versus the 7000 and that is exactly what I have found. This is the true irony of the current Nikon DX cameras. They have a much, much stricter lens sharpness requirement than FX cameras, and because of that are really more expensive to shoot with.

PB_PM

DXO Marks can be helpful, if you take the time to look at the information about what the different scores mean, but but I would not put too much weight into the results.

You are far better off trying a lens yourself, if it is possible to borrow from a friend or rent, than relay on the results of DXO marks.

Golf007sd

DXO Marks can be helpful, if you take the time to look at the information about what the different scores mean, but but I would not put too much weight into the results.

You are far better off trying a lens yourself, if it is possible to borrow from a friend or rent, than relay on the results of DXO marks.

+1

donaldejose

Everything is of some value in its own context. Imaging Resources used to have a feature which allowed you to directly compare the same photo taken by different cameras or by the same camera with different lenses.

manhattanboy

Everything is of some value in its own context. Imaging Resources used to have a feature which allowed you to directly compare the same photo taken by different cameras or by the same camera with different lenses.

Try here:

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx

spraynpray

I'm not convinced. I just looked on @manhattanboy's link and the 18-140 looked waaay worse than the old 18-105 at all apertures but the DxO mark test says the 18-140 is sharper. Who the heck is right?

We all know DxO is biased for Nikon, (don't we?) but I just don't understand how some credible people here can say how great the 18-140 is but DxO say it is the same as the lens it replaced. It was supposedly produced for the newest sensors, but whatever way you look at it there is no improvement if you buy it according to DxO.

Edit: Oh, and I am waaay past reading what I want to read in any test or review! 8-|

Fritz

Speaking of DXO- has any one had experience Optics Pro 9.5 as an editing program?

haroldp

@spraynpray:

"it is worse for CA, worse for distortion and worse for vignetting but better for sharpness"

The D610 might be worse than the D7100 in these respects with an FX lens because it uses the edges and corners which the DX crop does not.

I find KR to be a source of technically accurate info, from which he often draws wild and sometimes contradictory conclusions.

For practical and technically flawless, and useful analysis, and balanced explanations, Thom Hogan ( www.bythom.com " has the best site on the web, particularly for things Nikon.

.... H

spraynpray

Hi @haroldp, Yes, I understand the reasons for the less good results on FX, but I can't see how (on DxO) a lens which performs worse on an FX body in all areas other than sharpness gets such a high mark on the FX in comparison to the DX.

manhattanboy

Hi @haroldp, Yes, I understand the reasons for the less good results on FX, but I can't see how (on DxO) a lens which performs worse on an FX body in all areas other than sharpness gets such a high mark on the FX in comparison to the DX.

No one really knows how they calculate an overall score. Even their sharpness score is not really explained. Sharpness where? on the wide or tele end? at what aperture? None of these things are explained unfortunately. Hence why you have to take the sharpness with a grain of salt. Looking at Nikon's own MTF curves, the older 18-105 is sharper at the tele end and has better midframe sharpeness on the wide end, so I do not know who is saying it is much worse than the 18-140. The advantage of the 18-140 is the extra 35mm, and maybe updated VR, but not too much else. The digital picture's take on the 18-140 is terrible. But they only used an older DX camera. I have to wonder if they did not properly update the firmware and there was focusing issues as a result. It looks like they tried the 18-140 lens twice, as there is two instances in the database, again supporting the hypothesis that they recognized there was something wrong...they just did not know how to fix it.

I looked at the DXO interpretations for these two lenses. Basically they say there is no difference between the two on the 7100. Yes the new 18-140 has one more MP of sharpness, but likely that is because it is slightly better with stopping down. Interestingly the 18-105 appears sharpest wide open on the wide end, great for say indoor group shots, but probably not much use for a landscape shooter. For all practical purposes these lenses are about the same. Remember these days Nikon likes to produce more of the same and just call is something different LOL.

ads

DXO Marks can be helpful, if you take the time to look at the information about what the different scores mean, but but I would not put too much weight into the results.

You are far better off trying a lens yourself, if it is possible to borrow from a friend or rent, than relay on the results of DXO marks.

+1

For mine DXO and all other review sites are useful for sorting out what gear is worth considering, but I can't imagine ever buying a lens/body without shooting it and seeing real world shots.

haroldp

CA, distortion and vignetting can be corrected digitally, and often are in the camera (even on raw files), so it is hard to test the lens alone. Sharpness, (whatever that means) is harder to correct. Many lens designers today correct for sharpness (acutance contrast / resolution) and rely on digital corrections for other parameters, which explains why zooms today are so much sharper than they used to be.

I like the lens tests at imaging-resource ( www.imaging-resource.com )because they show you resolution at various focal lengths (for zooms) and f-stops.

I also do not fully understand dxo's rating parameters but agree with them that the Nikon 400/2.8 is the sharpest Nikon lens I own (and I have many) .

.... H

PB_PM

CA, distortion and vignetting can be corrected digitally, and often are in the camera (even on raw files)...

Nikon DSLR's do not correct CA and distortion in the RAW files, only baked JPEGS with said reduction tools active.

tcole1983

I didn't know about DXO prior to my lens purchases. I did read reviews by various places before purchasing what I got, but funds mostly made my decisions for me. I knew what I shot most and got the best I could afford. Like others I don't really understand how they rate some things like the overall score. I don't buy their ratings of lenses really...all mine on my D5000 are near poor, but in reality all my lenses are pretty sharp and good performers. Also they say my 35 F1.8 is better? Results from use don't seem to correspond to the results they show.

haroldp

CA, distortion and vignetting can be corrected digitally, and often are in the camera (even on raw files)...

Nikon DSLR's do not correct CA and distortion in the RAW files, only baked JPEGS with said reduction tools active.

Nikon does not, but others do. Fuji imbeds correction data in the raw file which is automatically applied by ACR. It can be defeated, but the default is to correct.

Nikon capture NX2 does not correct by default but will apply correction parameters if told to. I do not know if Nikon imbeds the parameters n the RAW files or NX2 uses a lookup table.

The point is that these digital corrections allow lens designers to optimize those around optical corrections that are not easily corrected digitally. How these lenses would perform on a film camera is an interesting question.

One day I might mount them on my F100 and try. The professional lenses will probably still be very good. There are no DX film cameras that can use the less expensive DX lenses.

... H

[Deleted User]

Speaking of DXO- has any one had experience Optics Pro 9.5 as an editing program?

I installed it, only because it has now a link with Lightroom. My first impression is, I get the same results as Lightroom, but .... I only did the basics. I 'am not waiting for another peace of software to learn and installed it only to try out things, this is my autumn and winter job. Till that time I sticl to LR and PS.

spraynpray

Soooo, in summary of what is above, is it reasonable to say that DxO is a Nikon friendly site and we use it because we like what it says? @heartyfisher has the 18-140 and rates it and @Pistnbroke says it changed his life, yet The Digital Picture shows it as rubbish (highly suspect IMHO as I have never seen any lens that bad) and DxO's results say it is just a tiny bit better. Come to think of it, they REALLY rate the Sigma 18-35 f1.8 as super sharp and give it high marks and you can actually easily see it is sharper just by glancing at the pictures from it. My own experience of the 24-120 was that it was noticeably bland and not sharp. I still don't know - maybe that means the newer Nikon lenses are as @manhattanboy jokes above - just something new to attract buyers. I see improvements in VR and autofocus, and differences in colour balance, but not the large difference in sharpness that we see with the Sigma 18-35 (the only new Siggy I have tried).

Msmoto

Like any commercial organization there will always be some mystery in how they tick. However, DxOMark provides one more set of data, when, combined with several others can be quite useful to a consumer looking to purchase/rent new equipment.

There is no end all to testing, and the final decision is in how the piece of equipment fits our own individual needs (or wants). I find DxOMark helpful, but also the consensus on NRF is also quite useful. For exotic (think 200mm f/2.0) items, an individual opinion can be nice, especially when used by someone who knows how to use it (think Squamish).

DxOMark can give us "in vitro" testing, but "in vivo" works best for my final decision making.

framer

The point is that these digital corrections allow lens designers to optimize those around optical corrections that are not easily corrected digitally. How these lenses would perform on a film camera is an interesting question.
... H

I have always thought to judge a lens from a digital camera there are too many variables. Looking under a 10X scope at a chrome taken from a test camera, on a test mount, with mirror up and remote release shooting a test target using test FILM, low ISO, is the only true test. Nobody does that anymore. The only problem is you still need to test several of the same lenses and consider statistical differences.

framer

PB_PM

They don't test with film for two reasons, 1) very few people still use it, 2) the way film gathers light is different. There are some inherent differences between film and digital sensors that would skew the results, such as the glass filters (UV/AA/etc) in from of the digital sensor. And frankly even film evolves over time, so to say that the results didn't change overtime is nonsense.

Nuke

I think DxO testing is relevant. I don’t like the way they round up certain features into one pretty number for a lens or a camera but I do like the individual tests they run on sensors and lenses. What site provides more information, in the areas they test?

DxO is great, so long as you only use it as one point of reference. There are many things they don’t test or comment on so you need other places, that look at the more physical things.

Besides, why complain? It’s free and you can decide if you want to use it or not. Adobe tests this stuff but what do you get out of them?