DxO Mark Ratings - are they any use at all?

PB_PM

Not to mention that lenses are only tested at maximum aperture, which for those who often use their lenses stopped down is also a meaningless measurement.

Paperman

@PB-PM
Are you sure about that as many lenses have worst sharpness at widest aperture. Why would DxO do that ?

haroldp

Most lenses (98 %) are sharper stopped down 1-3 stops, than wide open, only the very best are sharpest wide open, in the Nikon line these would be the 400/2.8, 200/2, 200-400/4 and possibly the 300/2.8 (which I do not own).

This is confirmed by very many tests by very many sources.

... H

PB_PM

@PB-PM
Are you sure about that as many lenses have worst sharpness at widest aperture. Why would DxO do that ?

I was referring to how DxO marks displays the rating. While DxO does test all apertures, if all you look at is the overall basic score, rather than the data, you could be mislead. After all, beside each lens they say "best at Fx.x", which is almost invariably wide open, regardless of the lens. Now, one has to keep in mind that rating is based on the "low light test." Again, something that could mislead those who don't take the time to look at the data.

Paperman

Nikon 50mm f1.8D is horrible wide open yet it says "best at f1.8" - it always indicates widest aperture as best . Yet, we know that is not true for most lenses.

The detailed measurement for the same lens page shows RED ( low resolution ) for wide open . Wish someone could explain that ...

spraynpray

Why? Because the full frame sensor allows more sloppiness on the part of the lens when transmitting the image compared to a crop sensor of the same megapixels.


Duh, I dunnunerstan: This talk of 'sloppiness' and megapretzels is making me dizzy. Surely having 24mp in the centre (best) area of a lens is going to give a better sharpness result (and probably CA but forget that for the moment) than having the same number spread right across the lens. Some lenses are visibly far worse than others on edge sharpness yet still get better sharpness figures on full frame.

Explain please.

manhattanboy

Duh, I dunnunerstan: This talk of 'sloppiness' and megapretzels is making me dizzy. Surely having 24mp in the centre (best) area of a lens is going to give a better sharpness result (and probably CA but forget that for the moment) than having the same number spread right across the lens. Some lenses are visibly far worse than others on edge sharpness yet still get better sharpness figures on full frame.

Explain please.

Is the relative detioration in edge sharpness twice that in the center ?
If yes, then there will be no difference. For any lens that is better then that, the larger pixels on the full frame allow better sharpness (or at least our perception of it). The difference in views really makes comparing crop to full frame difficult.
Nonetheless think about it like this: if I had a light particle and needed to get it into a box, how accurate does my aim need to be. If the box is bigger then there is more slop or leeway allowed in my accuracy.

spraynpray

@Manhattanboy: Hmmm, I think you are arguing the argument and not the point so to speak. I can't see anywhere in the results that explain what they are doing to arrive at their figures which supports your explanation. Interesting that the 'sides' taken in this discussion approximate the DX/FX divide - not withstanding those who have both, because they would still say FX would of course be better.

The fact that nobody has really clarified things and others have come on board similarly confused and skeptical shows that I was right to ask the question are DxO Mark ratings any use. My own experience says they are probably just creating sales of lenses and their own products because the differences they say are real don't seem to exist in reality. In fact, I would say I should believe my own eyes more and do what is often the advice here; hire/borrow one and check it out.

I'm still glad that I got the 70-200 f4 because its VR makes it more usable than the old 70-300 VR but the big IQ difference just isn't there. I had better add IME and IMHO now.

Where are @Ade or @TaoTeJared when you need them?

haroldp

I would propose an answer to this subject along the lines of:

DXO mark ratings are useful as one of many data points in evaluating a product, including a clear picture of ones mission for the product, and which parameters matter most to the mission and individual.

... H

SquamishPhoto

The fact that nobody has really clarified things and others have come on board similarly confused and skeptical shows that I was right to ask the question are DxO Mark ratings any use. My own experience says they are probably just creating sales of lenses and their own products because the differences they say are real don't seem to exist in reality. In fact, I would say I should believe my own eyes more and do what is often the advice here; hire/borrow one and check it out.

I'm still glad that I got the 70-200 f4 because its VR makes it more usable than the old 70-300 VR but the big IQ difference just isn't there. I had better add IME and IMHO now.

Maybe its your eyes? Has that crossed your mind?

Lets see some of your own test shots. How do both lenses fare at f4?

Bokeh_Hunter

My observations about the 50/1.4 D in the above post are 'factual' , the conclusions that one draws from them are opinion.

I agree with that - out of the 11 50's I own it certainly is not the dullest or the sharpest of the bunch for that matter. Some like the uber sharpness and can achieve it - most others need to work on their technique before any difference will ever be able to be seen.

Duh, I dunnunerstan: This talk of 'sloppiness' and megapretzels is making me dizzy. Surely having 24mp in the centre (best) area of a lens is going to give a better sharpness result (and probably CA but forget that for the moment) than having the same number spread right across the lens. Some lenses are visibly far worse than others on edge sharpness yet still get better sharpness figures on full frame.

Explain please.

Is the relative detioration in edge sharpness twice that in the center ?
If yes, then there will be no difference. For any lens that is better then that, the larger pixels on the full frame allow better sharpness (or at least our perception of it). The difference in views really makes comparing crop to full frame difficult.
Nonetheless think about it like this: if I had a light particle and needed to get it into a box, how accurate does my aim need to be. If the box is bigger then there is more slop or leeway allowed in my accuracy.

I would agree that DX bodies use the "sweet spot" of FX lenses and the overall field sharpness should show it to be sharper or at least equal-ish. (Given a comparison of 24mp DX & FX bodies.)

Take this comparison: 35mm f/1.8 (FX) on both the D610 & D7100. On the D610 it scores 11 points higher but it (due to the larger coverage area) shows higher Vignetting, slightly more distortion. Flip side, the D7100 shows worse CAs, much better vignetting, etc. Now the sharpness score is where I take issue with as obviously as we see from comparing camera's such as the D800 & D800e, resolution pulls scores wildly. But with two 24mp sensors, the sharpness changes. That makes zero sense in a rational way.

I personally did see the difference between a D300/s and a D700 where the D700 files were sharper. At the time the argument was "having larger 'wells' for light to be gathered in," made some since. To say that you would get a 3mp difference due to the relative small difference in well size, and knowing that basically the same processing of files (we are not talking about the difference between Canon and Nikon programming) it just doesn't pass the sniff test. That difference/improvement is adding something like 700-1,536 vertical lines (depending how you apply it) to the outputted file. (2,048w x 1,536h = 3.1mp) And that is at base ISOs. I have never heard that type of resolution difference between the two.
-----------------------------------

Bokeh_Hunter

There is always going to be an issue trying to measure an analogue device with a digital device that is limited and that is a given fact across a multitude of devices. There is also always going to be some differences between FX and DX bodies. We all know that. Basic comparisons between like items should be fairly close though.
If you have;
•the same resolution (24mp sensor) bodies
•from the same brand
•from the same generation (ie 7100 & 610 - not a D3s & D300)
•and the same lens which is being tested
Scores should be fairly close (when accounted for the difference of format, i.e. vignetting)

When long existing measurements (as in sharpness) are all of a sudden given a "new scoring" techniques, it is right and appropriate to evaluate them. If scoring is done correctly, correlations should be easily derived between like systems with static variables. Issues arise when these do not line up without any given explanation. And this is what we have with Dx0 - too many static or very similar variables and very wide gaps in the scores.

spraynpray

Maybe its your eyes? Has that crossed your mind?
Lets see some of your own test shots. How do both lenses fare at f4?

Erm, only one of the lenses can do f4 from 70-200...but yes, it had occurred to me that it could be my eyes, but as my friends lens is sharper than mine to my eyes, I concluded it wasn't.

@Bokeh_Hunter: +10 > especially to your last.

SquamishPhoto

Maybe its your eyes? Has that crossed your mind?
Lets see some of your own test shots. How do both lenses fare at f4?

Erm, only one of the lenses can do f4 from 70-200...but yes, it had occurred to me that it could be my eyes, but as my friends lens is sharper than mine to my eyes, I concluded it wasn't.

I was actually asking this of you quite seriously, despite Tao's rather brash intrusion into the conversation - so lets ignore him for now. Eyesight means a lot when questioning lens quality and even camera function. I was hoping you'd actually do some testing to prove your point, you seem like a pragmatic individual - and since Golf007 regularly does tests shots in his posts I imagined you might be inclined to try the same thing. I dig science, so it would nice to have something to judge besides opinion.

sevencrossing

In all seriousness
40 years ago I contacted Iritis, my GP failed to recognize it
I was working at Bristol Medical school at the time and fortunately my boss realized something was wrong. He marched me round to the eye hospital, they immediacy stared a course of atropine . but some of damage had already been done, my vision has never been quite the same
If you are in the slightest doubt about your eyes, it really is a good idea to get them tested

spraynpray

Like most people I tend to only get my eyes tested when I need new glasses, but when I do get them tested, I can always read the tiniest writing on the chart and the little card they give you to demonstrate your normal reading distance. Apparently, the text about three sizes up from the bottom is 20/20, anything smaller than that is better. I was also quite appalled to learn that they only guarantee 20/20 vision after laser surgery so I'm not going to pay out a load of money for a risky operation that could make my eyesight much worse, just to be told that it is still 'adequate' but I digress!

Bokeh_Hunter

...you get to choose how to waste it....

I enjoy having intelligent conversations and do not consider breaking down numerical calculations it a waste of time. I used to design and create metrics for businesses for a living. I quite enjoy backing into and breaking down numbers and find it quite easy and I enjoy sharing what I find. That was one of the tasks of that job I really enjoyed and still do to this day.

The thread is "DxO Mark Ratings - are they any use at all?" You have just attacked me on this thread and provided no insight on the topic at all. Your only contribution to the thread is suggesting someone should get their eyes checked.

I provide logic, details, and tests to show how I have come to my conclusions and where holes are in Dx0's results. Your argument has been, since I do not find Dx0 as the utmost and righteous entity, then Dx0 must be 100% correct and infallible. Oh and that they have a trademark of "DXO Image Science."

I am quite certain if I had not put TTJ in the signature you would not be commenting at all. I could have not disclosed it to avoid you and a few others continual attacks, but I am honest and don't see the point of not saying who I am. You have been spitting hate, spitefulness and disrespectful sarcasm at me and others for over 2 years. I will continue to call you out. There is a simple solution, stop attacking people and take your hateful comments elsewhere. Or you can actually provide unadulterated information to the discussions.

Forums are very simple things. They are discussions. If you are going to comment on a thread, it needs to be about the topic - not attacking people. If you do not agree with a position, then you provide specifics and details to support your stance. The continual spouting of your trolling and hateful character at me does nothing for the topic, and is a waste of everyone's time. No one thinks you are cute or funny and are just being a trolling prick.

donaldejose

Maybe NR should have a "time out" corner where some posters have to sit for a while with a dunce hat!

Personal attacks and name calling are not appreciated by most of us (some may get a laugh out of a witty "fight"). I just find it distracting to the subject at hand. However, we should remember this is a world wide forum involving many different personalities from different cultures in which different ways of relating are considered "normal" and "ok" so we should be tolerant of those comments which seem to be brash or arrogant or overly critical to us. They may not have been intended to carry the same emotion we read into them.

SquamishPhoto

I think Donald has the right idea. There is a difference between criticism and personal attacks and it really does seem like Tao can't distinguish between the two. Considering I was just called a "trolling prick" its safe to say that Tao truly does not realize how hypocritical he's being, which is unfortunate.

I provide logic, details, and tests to show how I have come to my conclusions and where holes are in Dx0's results. Your argument has been, since I do not find Dx0 as the utmost and righteous entity, then Dx0 must be 100% correct and infallible. Oh and that they have a trademark of "DXO Image Science."

No, you provide a logic that suits your opinion, which is not the same providing an argument with sound logic. An important distinction obviously lost on you. Also, no where in my statements do I give DXO anywhere near that kind of credit, this is just the delusion you insist upon being true. I criticized the nature of your argument since you seem to insist that DXO believes and advertises their numbers to be complete and totally infallible, which any reasonable minded individual can clearly see is not true. Providing the title of their trademark simple supports that assertion, namely that they are providing information on the imaging science of cameras. Any sane person sees that and understands that they aren't trying to cover all variables, but obviously we can't say that to you because it means that we're DXO fanoboys that bow to their holy grail. Im almost never on their page and have never let their metrics decide any lens purchase, but I don't have a love or hate for them. They are what they are and it doesn't make my blood boil or make me waste half my day writing painfully longwinded whinefests about how much I hate them and think they're wrong. That is your bent, so how about you just let it die already?

SquamishPhoto

Like most people I tend to only get my eyes tested when I need new glasses, but when I do get them tested, I can always read the tiniest writing on the chart and the little card they give you to demonstrate your normal reading distance. Apparently, the text about three sizes up from the bottom is 20/20, anything smaller than that is better. I was also quite appalled to learn that they only guarantee 20/20 vision after laser surgery so I'm not going to pay out a load of money for a risky operation that could make my eyesight much worse, just to be told that it is still 'adequate' but I digress!

Sounds like the right tact, I know a lot of people that are nearing retirement age that are avoiding the same procedure because their eyes seem to be working just fine with their eye glasses. And it sounds like you've actually got quite good vision, so I can scratch that off the list. I'd still like to see you do a comparison if that isn't inconvenient for you - owning one of the lenses being discussed makes me somewhat interested in what you'll find. And like yourself Im far too in love with the miraculously good VR to switch, but Im still curious to see how it fares in "real world" tests.

spraynpray

@Squamish: The problem is that I sold my 70-300 VR on ebay a couple of days after I got the 70-200 f4. I listed it Buy it now and it sold in 10 minutes.

I did some tests at 70, 135 and 200 at f5.6 when the f4 arrived and I wish I could say the difference was more obvious but colour balance and sharpness were VERY similar in spite of the supposedly higher spec of the f4. The main reason the f4 is performing for me is the super VR I reckon. sure, edge performance is better, but you know what? My mates Canon 70-200 f4 'L' series knocks it for six on colour and sharpness whatever DxO says (and I haven't checked but suspect DxO would be biassed towards Nikon).

PB_PM

My guess is that your lens/camera needs to be calibrated if sharpness is a problem. I have a hard time believing the 70-200mm F4 is no better than the 70-300mm VR at 200mm. Maybe at F8 they would be close, but at lower apertures? It's also possible that your copy has centring issues.

As for colour, that's down to personal preference. While I found the Canon to be very sharp when I tested it, I found the colours to be over the top.

spraynpray

My guess is that your lens/camera needs to be calibrated if sharpness is a problem. I have a hard time believing the 70-200mm F4 is no better than the 70-300mm VR at 200mm. Maybe at F8 they would be close, but at lower apertures? It's also possible that your copy has centring issues.

As for colour, that's down to personal preference. While I found the Canon to be very sharp when I tested it, I found the colours to be over the top.

Like I said above they were both tested at f5.6. Had to be because f4 isn't available at 135mm or 200mm on the 70-300 VR.

Msmoto

OK, guys, let's try to keep the posts on this and other threads as they pertain to the subject of the thread. Many of us have different experience, and for sure different opinions. And we can express these with all the data we have at hand.

However, there is simply no place for any personal assaults on another forum member. Please try to follow this rule.

Thanks

haroldp

@msmoto

Thank You.

... H