Automatic Exposure Compensation in raw processing

I’ve been experimenting with an ACES workflow using Canon .CR2 still images for awhile now, and I’m wondering if anyone can help me understand something fundamental regarding exposure with the scene-referred to output-referred process. (I’ve discussed this over e-mail with @cinelogdcp but haven’t been able to fully wrap my head around it.)

Let’s say I take a still on a Canon 5D Mark III with the Standard picture style and I get an in-camera .jpg (swiped from Imaging Resource) that looks like this:

(Original photo in sRGB color space, and resized in Photoshop using the embedded color profile with sRGB output.)

Then I take the corresponding .cr2 file into Photoshop’s Adobe Camera Raw, leave the settings at defaults, set the Camera Profile to “Camera Standard”, and get this:

(sRGB color space and 16-bit in the ACR import, output to 8-bit sRGB .jpg)

Very similar brightness and color.

Then let’s say I use dcraw to output an sRGB 8-bit .tiff file from the .cr2 using these settings:
dcraw -w -H 2 -o 1 -q 3 -g 2.4 12.92 -T Y_K2C1534.CR2
(Use camera white balance, blend clipped/unclipped highlights, use sRGB D65 for output colorspace, use AHD interpolation, set the gamma curve to sRGB instead of rec709, and output .tiff)

(.tiff brought into Photoshop, resized and output to sRGB .jpg)

Clearly a big difference from the first two. Now what if instead I use dcraw to output a 16-bit linear sRGB .tiff file using these settings:
dcraw -w -H 2 -o 1 -q 3 -4 -T Y_K2C1534.CR2

And I then load that .tiff into Nuke in a linear workspace with the OCIO aces_1.0.3 config, set the Read node’s colorspace to “Utility - Linear - sRGB” and the Write node’s colorspace to “default (Utility - sRGB - Texture)”, getting this .jpg:

Very similar to dcraw’s outputs vs. Canon’s/Adobe’s.

Now, I noticed with AMPAS’s rawtoaces tool that there is a “–headroom” option which seems to scale the luminance (?) of each pixel value, and using that I can get an ACES .exr that when viewed through the RRT+ODT looks like the same exposure as the in-camera .jpg and Adobe Camera Raw look, just with different color. But can anyone actually explain what that option is for and how it relates to the differing outputs shown above? I’ve seen some other posts here that mention adjusting exposure of an image converted to ACES in order to set middle-grey to 0.18, but I’m not sure if it is related.

To put it simply, if I shoot on a Canon DSLR, is the camera taking the linear input and applying just an sRGB gamma curve with some color adjustments, or is it always applying the sRGB gamma curve + some extra exposure? I totally understand that Canon and Adobe are making a lot of creative additions to the raw image, but I assumed that had much more to do with color hue/saturation than with exposure/gamma. And I also know that I may be mixing my terms here, hence part of my confusion.

Thanks for any help!

Holy cow, I think I finally found the answer:

I was not aware that Canon’s in-camera raw->.jpg processing and Adobe Camera Raw’s raw processing really were in fact always applying some exposure compensation behind the scenes on top of the linear to sRGB conversion. Apparently this is so that when you shoot a still based on the viewfinder preview and the jpg histogram, if your highlights look a little hot then when you develop the raw image on the computer it’s quite easy to recover them, since in the raw they probably aren’t clipped at all and there’s no actual reconstruction to do. This appears to be the case with quite a number of DSLR’s.

Sure wish it was some kind of option you could toggle in-camera, but at least I don’t feel like I’m crazy anymore trying to understand where my settings were wrong.

Yes, a good find. I had not read that before myself.

But also, I want to point out that
Standard in Canon JPG is not exactly a straight-line conversion to linear, whereas Neutral is as far as I can tell.
So I would expect a look difference in the JPG from the RAW.

I can confirm that we do a similar exposure compensation in Baselight/Daylight/Prelight for still images. I believe the exposure compensation origins historically from integer pipelines where you would not have the concept of setting diffuse white to 1.0.