How do Nikon DSLRs process Medium/Small image sizes?

TaoTeJared

The smaller in-camera JPEGs are downsampled from the RAW using a method Nikon will likely not disclose, but rest assured they are not "throwing away every nth pixel".

+1 I have never seen anywhere how or what method Nikon uses nor would I expect them too. Any other discussion on various ways to down sample images is just a fruitless academic discussion that has next to nothing to do with how Nikon does it.

I wanted to also feel like I was at least getting better signal:noise when I shot Medium or Small JPEGs and it sounds like I am.

Actually you are just loosing data - and not gaining anything. I have never seen Medium or small Jpegs have less noise than the large ones. By shooting Medium or Small Jpegs basically you have turned a $3,000 camera into a digi compact and are throwing away all the reasons to buy a D800.

Ironheart

@tc88, You are talking about demosaicing:
http://en.wikipedia.org/wiki/Demosaicing
And in a sense you are correct, but the end result of a demosaic is a RGB set of values for each pixel. This what is stored in the RAW file (hence the name). All other operations are performed after this step, so the JPEG of any size is created from the 36Mp of raw RGB values, not a different sample of the bayer (RGB) array. What you are describing is pixel binning which would result in a smaller RAW file.

Ade

I see that Wikipedia's EXPEED entry has a block diagram of the Milbeaut M-4 which indicates that the Large - Medium - Small "fixed" resizing in the image chain is using bilinear. Other algorithms including bicubic and nearest-neighbor are available for general resizing (e.g., for zooming in and out during image playback).


EXPEED Milbeaut M42 showing bilinear fixed resize, Wikipedia

The M-4 is the basis of Nikon EXPEED processors used in the D700, D3 and D3s. While not 100% conclusive, this information supports the notion that the in-camera JPEG resizing algorithm is some variant of bilinear, at least for Nikon cameras up to the D3s.

tc88

Ironheart, you are correct. I forgot that Nikon's raw is always stuck at the largest resolution. Thanks for the reminder.

Ade, I'm impressed that you always manage to find the relevent technical info.

roombarobot

@Ade, nice find, well done, you found our answer! Thank you.

Ade

We can't be certain of course, because Nikon might have implemented their own "special sauce" on top of the Fujitsu processor.

Now, even if we can't find out the actual algorithm, it is still interesting to know if there's any measurable noise improvement from selecting a smaller size JPEG. This is actually something we can try to test directly by using an Imatest rig or similar.

E.g.., in Imatest we can compare JPEG output vs. a reference color source (such as an X-Rite ColorChecker) and compute various noise characteristics. If we repeat the test several times for Large-, Medium- and Small-sized JPEGs of the same scene in the same lighting conditions, perhaps some conclusions can be drawn.

I'm "on the road" presently so I don't have access to my test rig but maybe this is something I can test in a couple of weeks.

roombarobot

We already know from basic principles of image and noise analysis that when averaging the Signal:Noise will go up by the square root of the number of noise-independent measurements:
http://en.wikipedia.org/wiki/Signal_averaging

One can, of course, get the same thing in post processing by doing a similar average. You do also, of course, lose information, specifically resolution. However, this is good for my purposes, it helps me get my gear out more and reduces the post-processing I don't like to do.

Ade

Yes at first glance the noise should be improved due to averaging since noise in adjacent, independent pixels should cancel out each other.

However, as the M-4 block diagram shows, for JPEGs the resizing is performed far down the image processing chain. By the time resizing happens, the sensor data is already demosaiced, gamma corrected, color corrected, filtered (e.g., sharpened), and perhaps gone through one or two passes of noise reduction.

So I'm less than convinced that we can further improve SNR of highly processed images by averaging, since the noise components of adjacent pixels would no longer be independent of each other.

roombarobot

So I'm less than convinced that we can further improve SNR of highly processed images by averaging, since the noise components of adjacent pixels would no longer be independent of each other.

Very true, if the noise is no longer random and independent, then it will not cancel itself out. At least it isn't just dropping pixels, that's great to know.

roombarobot

I wonder if the new RAW Size S on the D4s is binning? It is using 1/4 the pixels, so it could pretty simply bin them, if the Expeed 4 and the firmware supported it. Too bad it is 12-bit.


RAW S Small* (12-bit uncompressed) image size option
A new RAW S Small option that records images using 1/4 the number of pixels used for full-sized RAW images has been added. This makes editing images on a computer after they have been taken faster and more convenient (file size is approximately 1/2 that of 12-bit uncompressed RAW L Large images).

Ironheart

Sounds like it. There should also be an increase in usable ISO if a true HW binning is being performed.

PB_PM

Why bother on such a low megapixel body like the D4s? They should have brought that feature in with the D800/D800E.

Ade

Workflow speed and storage/transmission volume. Many PJ type assignments are so time sensitive that even at 16mp the photographer is forced to shoot only JPEGs.

With RAW S, both the storage requirement and speed of the NEF processing should be about the same as a JPEG workflow. It will be a benefit where the photographer knows that some post-work may be necessary (e.g., in mixed lighting situations, when shooting high contrast outdoor events, etc).

D800/E shooters go in for the full 36mp glory and implicitly understand that massive file sizes mean beefier computers, tons of storage, slow workflows and long upload sessions. Time is usually less of a constraint compared to (say) the agency sports photographer working under a tight deadline.

It will be interesting to test how well RAW S is implemented. I suppose this will be a standard feature for the prosumer line and up.

roombarobot

I would love if a firmware upgrade could bring binning to my D800E. How about it, Nikon?

Ironheart

Not unless you want to open up your camera, desolder the Expeed 3 chips and replace them with Expeed 4. I'm pretty sure this is a feature of Expeed 4. Now maybe on a D5300 or D3300 a firmware update could add this, if there were enough demand and Nikon saw value in it. Naw, never gonna happen...