Strange 50MP phenomena to expain...

Pistnbroke

I don't know how many do this but to compare cameras using DXO I pick a sharp 3rd party lens and look at the sharpness rating on each camera ...
eg
D700 .....11 MP
D800......20MP
D810......27MP
So you would expect a Canon 5DSR ( a 50MP camera) to be what?? 40MP...no 18 MP

clearly the lens is sharp enough to generate 27MP so why so low ...did Canon fire too soon? (joke)

heartyfisher

Which lense ?

Pistnbroke

13mm Samyang

spraynpray

Interesting. What aperture was that test done at?

Pistnbroke

Well perhaps its me firing me Canon too soon..I checked the Sig 24-35 and the 810 gave 26MP and the D5r 32......oops Tried the 85mm Ziess and got 35/41MP oops again but the 85mm Samyang gave 22 and 18. What I don't understand is why the MP falls with a lens capable of producing more than the D5s rating.
Perhaps this says something about Samyang but I am not sure what.

heartyfisher

So the primary criteria is ok then? the 50mp is better than the 36mp :-)

Ironheart

Now you know why some folks hate DxO. Their scores can be a bit of a mystery. They only test a small sample as well.

donaldejose

Very interesting. Must be some reason and I doubt it would be the lens. Consider the Sigma Art 35mm f1.4. On a Nikon D810 if produces a perceived megapixel rating of 30 which is 6 less than the 36mp sensor. On a Canon EOS 5DSR it produces a perceived megapixel rating of 34 which is 16 less than the 50 mp sensor. Seems like Canon sensors are underperforming relative to Nikon sensors for some reason. Perhaps when Nikon produces a 50 mp sensor we will have a more direct comparison.

WestEndFoto

Very interesting. Must be some reason and I doubt it would be the lens. Consider the Sigma Art 35mm f1.4. On a Nikon D810 if produces a perceived megapixel rating of 30 which is 6 less than the 36mp sensor. On a Canon EOS 5DSR it produces a perceived megapixel rating of 34 which is 16 less than the 50 mp sensor. Seems like Canon sensors are underperforming relative to Nikon sensors for some reason. Perhaps when Nikon produces a 50 mp sensor we will have a more direct comparison.

I am sure that the Canon sensor is performing just fine. It is likely that the Sigma Art 35mm f1.4 is only so sharp and you are seeing limited returns with higher megapixel sensors. 34 or slightly more is likely the maximum resolution that will ever be achieved with that lens, regardless of the sensor. This is of course using the DXO methodology, which I believe is an average across several aperture settings. It likely performs better at f/5.6 (where it is likely diffraction limited - so great lens) and worse at f/1.4.

I think as we see higher megapixel sensors released, we are going to see this more and more. It is why I pay more attention to lenses than I do cameras.

My understanding of the DXO rating is that it is an average over many aperture settings. If this is true, the higher megapixel sensors will likely provide a more significant improvement at f/5.6. I wouldn't through out the Canon just yet.

The Samyang result is interesting though. Hmmmm........

Ironheart

The "diffraction limit" is when the diameter of the Airy disk's central peak becomes too large relative to the pixel size in the camera. Once two airy disks become any closer than half their width, they are also no longer resolvable, this is called the Rayleigh criterion.

If you compare the size of the Airy disk to the pixel size (pitch) you can find which aperture diffraction starts to set in. For a 36mp FF (FX) sensor it starts somewhere between f/8 and f/11. For a 50mp FF it's between f/5.6 and f/8.

Perhaps the DxO tests use f/8 as one of the test apertures, and we are seeing that in the results. Or they are just a load of crap

donaldejose

Interesting.

haroldp

DXO ratings are so imprecise being a single number weighting multiple conditions and factors, that I have no idea what they mean.

lens tests from imaging resource are much more useful .

Pistnbroke

Pleased you don't think I am mad ....
Looking at the Tam 28-300 the 810 and 5DSR give 10 MP so I deduce that's the best that lens can do but the Sigma 28mm gives 810 23 and the 5drs 22....same scenario.
Sigma 85 mm 810 23 and 5DRS 33
Sam 85 810 22 and 5DRS 18

In general I am not seeing a consistant improvement with the 50MP sensor.
The sharpness graph for the Tamron 150-600 is weird if not impossible for the 5DSR.
Lets see what Nikon does !!!

spraynpray

I reckon its a good time for Nikon to concentrate on even better hi ISO.

paulr

Spraynspray I think they have done that with the D5.

Paperman

I mostly favor DxO and trust their scores for Nikons but they do occasionally have findings like this against common sense/knowledge.

In this case, it is probably different samples of 3rd party lenses on different cameras ( since they can't be keeping all the lenses they test for years ) and some samples are simply worse than others.

What surprises is me that DxO still publishes these results even though anyone ( including the ones doing the tests ) can see the inconsistency. No second brain there saying " hey, wait a minute - sth gotta be wrong here. Let's get a different sample " without going ahead and disclosing the scores ( without any side marks on inconsistencies ) :-?

WestEndFoto

Pleased you don't think I am mad ....
Looking at the Tam 28-300 the 810 and 5DSR give 10 MP so I deduce that's the best that lens can do but the Sigma 28mm gives 810 23 and the 5drs 22....same scenario.
Sigma 85 mm 810 23 and 5DRS 33
Sam 85 810 22 and 5DRS 18

In general I am not seeing a consistant improvement with the 50MP sensor.
The sharpness graph for the Tamron 150-600 is weird if not impossible for the 5DSR.
Lets see what Nikon does !!!

I see nothing out of the ordinary here that causes me to wonder about DXO's methodology. Some lenses are mediocre and produce the same results regardless of the resolution of the sensor. Some are a little better and a better sensor offers a significant improvement.

And then some see a slight reduction with the higher megapixel sensor. That does not surprise me as their could be a variety of reasons. The most obvious reason would be sample variation in individual lenses. DXO is susceptible to that as they only use a sample size of one (a critical error in my view that the Lensrental guys pointed out). However, you may actually get different results between Canon and Nikon. Remember that Canon lenses and cameras are optimized for each other and the same for Nikon. It is hard or even impossible for a third party supplier to optimize for more than one camera. You may actually get slightly worse results with the higher resolution sensor for this reason alone. And remember, the difference between "22" and "18" is actually quite small.

BVS

My understanding of the DXO rating is that it is an average over many aperture settings. If this is true, the higher megapixel sensors will likely provide a more significant improvement at f/5.6. I wouldn't through out the Canon just yet.

For prime lenses the DXO sharpness score is for the sharpest aperture of the lens. No averaging. For zoom lenses they average the sharpest apertures of the various focal lengths measured.

http://www.dxomark.com/About/Lens-scores/Metric-Scores

The first sentence in the Sharpness section makes it sound like they average all the apertures, but if you read the whole thing it says they take the sharpest apertures at each focal length and then average those together. Since a prime lens has only one focal length the score is thus the sharpest aperture of the lens.

It's actually possible to test this by finding a prime lens/sensor combination that has a very low sharpness score, like 6 to 8 or so so it's an identifiable color on their graphs, and comparing the graph to the score.

Also, they weight center performance more highly than edge performance, which is probably (at least one reason) why the new 24-70E ended up with a worse sharpness score than the the 24-70G.

Here's some other interesting pages I ran across that describe their testing process:

http://www.dpreview.com/articles/3135549652/lens-reviews-explained/7

http://www.dxomark.com/About/In-depth-measurements/Measurements/Sharpness

http://www.dxomark.com/About/In-depth-measurements/DxOMark-testing-protocols/MTF

heartyfisher

I like DXO .. :-) I think they don't play games or exaggerate. I belief that if there are "inconsistencies" Its either 1) I mis-understood, 2) a limit to their "well enough" documented methodologies. 3) Lense testing is a complex issue and numbers do round(3.0 = 3.4 = 2.5 ? ). and criteria sometimes compound. I have not been interested enough in Canon stuff to look and analyse the DXO findings so I cant explain your discoveries... but I have looked at the Nikon data in quite alot of detail and have even been able to deduce non standard "stuff" from the data so I do belief that it is fairly consistent especially when considering multiple data point across multiple sensors/ lenses/ generations

Pistnbroke

I understand that DXO don't test all the lenses with all the cameras and mathematically interpret the results from past data. Would you want to test new lenses on a D70 or D700
I was of the opinion they were a USA company but I understand that is not so.

spraynpray

Interesting - I wonder at what point a body becomes 'old' and the numbers aren't measured then?

donaldejose

I would think they do not keep all bodies of all cameras in stock to test a new lens on. I would expect that they develop a relative scale of old body to new body, test the new lens on the new body and then use the scale to report a relative number for that lens on an old body. One would think they have to do something like this to reduce the amount of inventory held and the amount of testing done. Anyway, who is going to buy a Sigma Art 50mm f1.4 and put it on a Nikon's 6mp D70 body?

Ironheart

Anyway, who is going to buy a Sigma Art 50mm f1.4 and put it on a Nikon's 6mp D70 body?

Why not?
DxO is dual headquartered in San Franciso and Paris from what I remember. It's on their website.

heartyfisher

Interesting - I wonder at what point a body becomes 'old' and the numbers aren't measured then?

Huh ? Unsubstantiated assertions... please .. my hair cant take it ... ~X(

I recently shot with my newest lenses on my 2 6MP oldies ..

Pistnbroke

Well if I can't stop me 54MP down to F8 we are going to have a lot of out of focus wedding pictures !!

spraynpray

Strangely, the 36mp that I always thought to be more mp than I want is becoming more appealing with the thought of a 54mp monster arriving. O:-)

54mp will not be for me. What will macro shooters do without f11 available to them - stacking is not feasible with moving subjects and mp will not make unsharp images sharper...