DxOMark trashes Leica M9, "worst image quality"

Ade

From our sister site:

http://leicarumors.com/2013/03/06/first-leica-camera-test-results-published-at-dxomark.aspx/

"In fact, with a DxOMark Overall Score of 68, or 69 for the Leica M9, M9-P and ME Type 220, these cameras offer the worst image quality DxOMark have tested on a full frame sensor, with the exception of the 10-year-old Canon EOS 1Ds."

Godless

So, Leica has a worse sensor than the other cameras. That´s why it always needs the f/1.0ish glass or a lot of PP afterwards..

haroldp

I heavily use both Nikon and Leica (for 45 years) and at the current or prior generation, (M9, D800, D3x) are all capable of of IQ at a level where IQ is no longer the issue that matters. I have no F1 glass and my most frequently used Leica lenses max at F2 or 2.8 (I have an F1.4 Summilux but use it less than my other lenses).

The M9 used within it's envelope (ISO <= 800 etc.) with no AA filter and Leica glass is capable of stunning IQ.

MF sensors do not even go that high with any quality, and color film over 400 was awful.

Sensor designers (including firmware) make tradeoffs and ISO capability vs DR and color depth is one of them assuming the same quantum efficiency.

I would not use Leica for sports, wildlife, near darkness, or anything moving fast, my AF and VR Nikons with high ISO capability are simply better for that purpose.

60-70 % of my work does not fall into those categories.

For candid street shooting, narrow DOF (even Zeiss glass is no match for a Summilux at F1.4) and other types of city, or landscape shooting, Leica is excellent.

In North America at ISO 320 in bright sunlight, at f4, the exposure for front lighted subjects is a 1/5,000 sec
in cloudy overcast either 1 /1000 sec or 1/500 sec.

The IQ of any modern equipment in this class when well used, is so good that 'potential' IQ is rarely the decision point in equipment choice. Other factors take precedence based on expected usage.

Regards .... H

Benji2505

I agree with Harold. DxO can only comunicate the measurable IQ. A lot of IQ comes from the Leica M lenses though.

TaoTeJared

What I have seen of the M8 & M9 is the color gamut is much wider than sRGB, Adobe and beyond Nikon and Canon sensors by a great deal. I do wonder how much that plays into it.

Just playing around to see "how it compared" to others...



By their scores - the Leica is no better than the m4/3rds cameras. I don't think so. Fails on the "sniff" test.

haroldp

What I have seen of the M8 & M9 is the color gamut is much wider than sRGB, Adobe and beyond Nikon and Canon sensors by a great deal. I do wonder how much that plays into it.

Just playing around to see "how it compared" to others...

You have hit an important point.
Leica designers have chosen to expand the gamut at the expense of high ISO performance.
This is exactly what the medium format producers have done and accounts for the 'look' of MF (and Leica) . It also accounts for MF sensors twice 35mm size being hopeless at ISO 400.

Neither trade-off is right or wrong, fitness for the mission is the key question.



By their scores - the Leica is no better than the m4/3rds cameras. I don't think so. Fails on the "sniff" test.

TaoTeJared

I'm not sure if the "expand the gamut" has anything to do with the noise. Much of that has to do with them using a CCD (Leica and MF) instead of Cmos (although the new M uses CMOS sensors). The design differences of CCDs inherently have more noise vs CMOS and CCDs at native isos are known to be a bit better although that gap has closed considerably. I know some were looking forward to the CMOS sensors as they could have increased the headroom being able to use higher ISOs but that doesn't seem to have happen. Considering the uses for the Leica M, you would think they would focus on getting the noise under control and clean to 1600 and good enough to 3200.

Either way - for the scores to indicate they are no better than M4/3 is a joke.

spraynpray

Perhaps it is more of a failing of the DxO test/marking system than the camera itself?

ads

"What I have seen of the M8 & M9 is the color gamut is much wider than sRGB, Adobe and beyond Nikon and Canon sensors by a great deal."

So does that gamut show up in prints @TaoTeJared? Obviously a gamut that wide won't show up on your average monitor...

JJ_SO

A wide gamut can't be printed "as it is". But it can be compressed to something printable on a high end printer. Meanwhile, if there's a smaller gamut, I would loose a lot of tones right in the moment I'm trying to catch them. And not always HDR is doable, if your target is moving.

ads

Cheers @JJ_SO, that makes sense

Ade

Having a much wider gamut than what can be viewed or printed is just a waste of bits, so no, the color gamuts of Leica M8/M9 cameras are not "beyond" Nikons/ Canons by a great deal. In fact the D800 can reproduce more colors than any Leica Ms.

Most sensors today have a gamut somewhat comparable to AdobeRGB, exceeding it in some areas but have gaps in other areas.

TaoTeJared

"What I have seen of the M8 & M9 is the color gamut is much wider than sRGB, Adobe and beyond Nikon and Canon sensors by a great deal."

So does that gamut show up in prints @TaoTeJared? Obviously a gamut that wide won't show up on your average monitor...

It can show the entire gamut in prints with very high end printers. What you can do is "shift" the color pallet and even the newest printer with the high number of inks can display well beyond even what the CCDs capture.

Having a wider gamut is only a waste if those who are using the files choose to waste it, compress it, change it to AdobeRGB, sRGB. To believe that the eye can only see AdobeRGB or any other of the 100s of color profiles is ridiculous. We can perceive an infinite amount of color variation.

TaoTeJared

Perhaps it is more of a failing of the DxO test/marking system than the camera itself?

I think this is the case. There has been a slug of cameras who's scores when compared to each other just don't make since.

Msmoto

Maybe this just shows that any opinion of performance is still limited to what one person things is important and may not be the same for all of us. I find noise not much of an issue as I came from the time grain might be like gravel. Yet, younger folks sometimes think grain/noise is an absolute failing.

DxOMark is looking at a few parameters and the Ernst Leitz people may be looking at different ones.

haroldp

Maybe this just shows that any opinion of performance is still limited to what one person things is important and may not be the same for all of us. I find noise not much of an issue as I came from the time grain might be like gravel. Yet, younger folks sometimes think grain/noise is an absolute failing.

DxOMark is looking at a few parameters and the Ernst Leitz people may be looking at different ones.

Thank you.

We sometimes obsess about one aspect of performance, when different aspects vary in importance depending on the mission and what one cares about.

I also remember when the fastest practical color film was ISO 400 ( 800 was almost not worth using as it had visible grain on a 35mm 8 X 10 ) and it was horrible by the standard of any modern DX or FX format camera.

Getting exposure and focus 'right' and of course lens quality, dwarfs any inherent differences in IQ of modern sensors.

Regards .... H

Ade

There are opinions, facts, and also myths.

Fact: plot the M9 color gamut vs. AdobeRGB and anyone will find that the M9 gamut is not wider "by a great deal" -- if it is any wider at all. So much of the above discussion about high end printers, compression, etc., is just irrelevant and misleading.

See M9 gamut chart here:

http://www.amateurphotographer.co.uk/reviews/dslrs/129204/8/leica-m9-review

Quotation: "Like most digital cameras, however, the M9 cannot quite replicate the full range of delicate green shades in the Adobe RGB colour space."

Godless

Perhaps it is more of a failing of the DxO test/marking system than the camera itself?

I think this is the case. There has been a slug of cameras who's scores when compared to each other just don't make since.

I think you´re wrong on that point. Leica M9 gets old, and the new m4/3 cameras/sensors are really outstanding especially when it´s Leica FF vs a tiny m4/3 sensor. But hey, that´s sensor evolution for you..

And if any other old top-shelf camera can be caught up by the newer tools, why couldn´t Leica?

I believe the DxOmark is the most accurate camera sensor comparison tool to date.

TaoTeJared

There are opinions, facts, and also myths.
Fact: plot the M9 color gamut vs. AdobeRGB and anyone will find that the M9 gamut is not wider "by a great deal" -- if it is any wider at all. So much of the above discussion about high end printers, compression, etc., is just irrelevant and misleading.
See M9 gamut chart here:
http://www.amateurphotographer.co.uk/reviews/dslrs/129204/8/leica-m9-review
Quotation: "Like most digital cameras, however, the M9 cannot quite replicate the full range of delicate green shades in the Adobe RGB colour space."

When you test a camera from just shooting Jpegs - I would agree - the output is significantly handicapped by the format. Diglloyd has the gamut if you subscribe to his leica subscriptions. It is very close to Luminous landscapes M8 profile.

I believe the DxOmark is the most accurate camera sensor comparison tool to date.

At one time I would have agreed - but now there is something seriously broken. Reviews on their software is starting to take a big hits as well on color accuracy, and sensor profiling. Their "tests" came to be because they did them for their software updates and go hand in hand. There are many other companies out there that do profiles and "tests" but for software editing, not to sell their name based on their tests.

haroldp

Everybody's sensors today are more efficient than the prior generations, and competent engineers can work the trade-offs several ways.

The M9 sensor is 4 years old, and cannot be compared to sensors 1-2 design generations newer.

The M240 is reputed to be about 1 EV better.

That being said, even the M9, D700 / D3 etc. are good enough that for most conditions, the sensor is not the primary determinant of photo IQ.

I do however use my D800e for theater work because I need clean ISO 6400 for those conditions.

I also still use the D700, D3x, and M9 for the vast majority of my situations where ISO 400 is fine.

Regards ... H

As engineers we like to quantify and measure, we also sometimes attach more significance than deserved to those things we can measure simply because we can measure them.

It reminds me of the good old days when audio components were only measured by frequency response and THD and it was pronounced that those measurements full defined the sound even though identically measuring components sounded very different.

While I am sure that ultimately almost everything is subject to measurement, that which we can and do measure is not always more significant than that which we do not yet know how to measure, or is simply to difficult.

While DXO measurements are themselves neutral, the weighting of factors going into a final one number 'score' is very subjective, whether it be sensors or lenses.

How to weight noise vs resolution v size and magnification.

How to Weight lens resolution center v edge, at what F-stop, v CR (lateral or axial), v distortion, v astigmatism, v flare, v contrast (center v edge) etc.

If software easily corrects distortion and some abberations do we care ?.

How does CR show up as lost resolution before / after correction ??.

The idea of creating a composite one number aggregate 'score' for these factors absent usage criteria is worthy of consumer reports, and not at all useful to me.

A detailed description of the above properties which I could then evaluate against my own needs and priorities is much more useful to me.

I like imaging-resources lens tests for this reason, and when augmented by sample varisation data from lens rentals, is much more useful than a 68 v 69 aggregate.

.... H

Ade

When you test a camera from just shooting Jpegs - I would agree - the output is significantly handicapped by the format. Diglloyd has the gamut if you subscribe to his leica subscriptions. It is very close to Luminous landscapes M8 profile.

Um, obviously that gamut diagram is from M9 RAW, not JPEG.

Godless

The DxOMark only measures measurable quantities and based on their tests they give a score to a sensor.

So many people tend to take that as a personal hit and misunderstand it the way that certain equipment absolutely cannot take good enough photos. A one point difference in DxOMark scoring won´t mean a thing, but ten points in the overall score may do so in EXTREME conditions. You will not see a difference in family portraits in normal lighting conditions. I am mostly interested in their Landscape and Sports ISO rankings, which tell me, whether I will like a certain sensor´s output at the ISO values I need for my stuff.

I am glad that the Leica-hype is finally measured for what it really is. I doubt this will affect the pricing in any way (unfortunately), but it´s nice to know that my gut feeling was right about it. For the price of one Leica lens, I got a full enough m4/3 system that takes surprisingly good images in low light.

TaoTeJared

I am glad that the Leica-hype is finally measured for what it really is.

And that sums up the conversation and why "it must have to be true" and want to it to fail - affordability. The most expensive system "must" not be as good as people say - it's only hype.
Some are predisposed to look past evidence that is staring them in the face to justify their foolish animosity towards something that they can't afford. If you are unwilling to step back and detach your emotions of envy, you should focus on other threads.

There is nothing personal about it on my end - in my endeavors to gain an understanding of what makes great images and the tools people use - I have never seen a m4/3rds come close to a Leica quality. Most of the medium format systems score this low as well - and you can't convince anyone M4/3rds have outdone those.

TaoTeJared

Those who don't have an indoctrinated belief that DxO is the Divinity of camera tests and are more inclined to deductive reasoning - pull up any camera with 18mp +/- and look at the scores.

Pick any 16-18mp MF, DX, FX, M4/3rds camera and look at the results- kind of interesting how the scores are grouped? Then do that with 24mp.

If the M220 and M240 have identical sensor manufactures, base designs, software and the only difference is 18mp vs 24mp - effective vertical resolution is only 15% (30% total) how is the low noise somehow over double or a full stop better with the higher MP. All of the scores are a bit better - but at the base, the design is the same and so is everything else but the pixel count.

We have not seen any other company release the same system with two different sensors at the same time before to really have an apples to apples comparison. Why some feel the need to attack anyone who points to the rather illogical results is beyond me.

ads

"Most of the medium format systems score this low as well"

The DXO ratings we are discussing here aren't rating systems, just sensors. When you are talking about real world results you are including the lens in the equation which is another very significant variable.

Don't forget medium format sensors are rarely tuned for high ISOs, and that is one of the primary factors in DXO's ratings, so they are never going to fare well on their overall scale. Not doubt the Leica is in the same boat. You just need to look past the single overall number to get a proper picture for how they will perform for the task you want to use them for.

Ade

Don't forget medium format sensors are rarely tuned for high ISOs, and that is one of the primary factors in DXO's ratings, so they are never going to fare well on their overall scale.

Except that they do. In fact before the introduction of the D800s last year, medium-format sensors dominated the top DXO ratings.

And even today there are three MF sensors in the top 10, with the PhaseOne IQ180 is ranked #5, beating all Canon DSLRs.

DXO has yet to review the latest MF sensors (e.g., IQ280), so MF might again reclaim the top positions once they do.