Output Transform Tone Scale

Discussions - ACESNext VWG – Output Transforms
55

Most CAMs I have seen have a fixed adaptation point which makes them not terribly well suited for moving imagery? Are there more fundamental mechanics we could start with?

56

Great discussions about theory @Troy_James_Sobotka and @SeanCooper, this is great and I hope it continues.

I will now interject with a bunch of boring math. Boring but useful, hopefully.

Musings

I spent a while investigating approaches. Handling intersection constraints for the display-referred “toe” flare model is tricky. After a couple of weeks working on the problem, and some help with the math from my physics major friend, I’m pretty confident that there is no closed-form solution for a grey and white intersection constraint including the toe function as Daniele proposed it. (Happy to be proven wrong here by someone smarter!).

I am also wondering if it is the correct thinking for the toe to have an intersection constraint at grey. Because when it does, contrast is changed when you boost the toe value. Might be an unwanted behavior. Maybe it should be treated more as a surround compensation power function, where the mid-grey point is allowed to shift?

Siragusano Variations

So far I’ve come up with 3 potentially useable variations of @daniele’s model.

  • The first is super simple. Just the compression with contrast, solved for grey and white. It could be used as a module if another model was desired for surround or flare compensation.
    Tonemap_Siragusano01.nk (1.3 KB)

  • The second is a lot more complex. I managed to solve the toe intersection constraint through grey by solving in terms of a second power function applied on the output of the toe function, which is also used for unconstrained surround compensation. The curve gets a bit weird at higher t_0 values, but seems to work okay at more common low values, and also seems to increase contrast a bit less than the alternative of solving in terms of s_0 (which also does not seem to be possible using analytical approaches).
    I dislike the complexity required to get such a mediocre result.
    Tonemap_Siragusano02.nk (2.0 KB)

  • The third is basically the same as the first version I posted, which uses the intersection constraints from the first simple version above, and decouples the toe compression from the grey intersection constraint entirely, while pivoting at r_1, so that boosting toe compression does not lower the peak white luminance.
    Tonemap_Siragusano03.nk (2.0 KB)

There is a summary and some more verbose info in this notebook.

Piecewise Hyperbolic

I got a bit fixated on the idea of having a linear section below the grey intersection point. In the testing I have done I like the way it looks in images, as I mentioned in my previous post. Unfortunately the hyperbolic compression curve formulation I knew from the gamut mapping vwg got super complex trying to solve for the intersection constraints.

So I decided to start from scratch and build my own Hyperbolic compression function. This endeavor being relevant to my ongoing fascination with conic sections…

So I started with f\left(x\right)=-\frac{a}{x} and ended up with a relatively simple formulation that works well. Here are a few variations:

There is a more verbose derivation of my custom hyperbolic compression function and some more details in this notebook.

2 Likes
57

After playing around a bit with all of these different variations, I decided that I did not like the behavior of Toe constrained by the grey intersection point. Changing contrast when adjusting toe doesn’t make sense.

I settled on using the piecewise hyperbolic with “decoupled toe and surround”. I played around a bit and roughed in a few presets for

The behavior and appearance is nice but I don’t like how complex it still is.

I keep going back to @daniele’s original formula, and thought I would give the same exercise a try with his original “minimally constrained” formulation and see how it went. It was actually surprisingly easy to dial in the same behavior, even without being able to specify the precise intersection points. (I did make one modification in order to get the output domain scaled to the right level for PQ output).
Tonemap_PiecewiseHyperbolic_Presets.nk (4.0 KB)

Edit: Woops! I realized looking back at this that I uploaded the wrong nuke script here. Here’s the correct one which contains Daniele’s compression function, as described above.
Tonemap_Siragusano_Presets.nk (2.5 KB)

Which leads me to an observation that I probably should have had before spending a few weeks on the above math exercise: In a scenario where we are building something that has preset parameters for different displays and viewing conditions, is a high level of complexity in the model justified in order to achieve precision of intersection points? What level of precision is necessary here and why?

Edit:
I realized I may not have shared the super simple EOTF / InverseEOTF setup I’ve been using, which may be a necessary counterpart for testing the above tonescale setups, especially with HDR:
ElectroOpticalTransferFunction.nk (5.2 KB)

3 Likes
58

That’s a very good point, @jedsmith. I don’t believe we are trying to build a fully dynamic, user controllable display transform. So as long as we can find values that create the desired intersections for a set of common targets, and clearly document a methodology for calculating them for other targets, it is probably not necessary to restrict ourselves to an algorithm where the intersections can be solved “on the fly”.

3 Likes
59

Hi @jedsmith ,

I’m testing your latest tonemaps on some images and wanted to make sure I had things set up correctly. I’m on a standard sRGB monitor, I have the image set to ACES-2065, the Nuke view transform set to Raw, and am using the Display EOTF node from one of your scripts, set to sRGB. Here’s a screenshot of Nuke:

Nuke1
Nuke11404×900 344 KB

As you can see, this seems to appear somewhat desaturated. On a hunch I added in a ACEScg to sRGB color matrix from one of your earlier Nuke scripts (20190119_tonescales.nk). Here’s a screenshot of Nuke:

Nuke2
Nuke21404×900 414 KB

Visually this makes it match the ref image. However, I suspect it’s not quite right, as the matrix is for ACEScg to sRGB and my image is ACES-2065. Here’s the matrix code:

ColorMatrix {
matrix {
{1.705079317 -0.6242337823 -0.08084625006}
{-0.1297003478 1.138468742 -0.008767957799}
{-0.0241663754 -0.1246140897 1.148780584}
}
name ColorMatrix10
label “ACEScg to Rec709”
note_font Helvetica
selected true
xpos 102
ypos -789
}

Apologies if I’m doing this completely wrong here. Happy to be pointed in the right direction.

60

Hey @Derek !
Thanks for getting your hands dirty. We need more people testing things out!

Modules

The stuff I’m posting in this thread is purely related to the “tonescale” component of the display transform. While important, it is only one component. A potentially incomplete list of different modules we might need:

  • Input Conversion - Convert input gamut into the rendering colorspace, in which we will “render” the image from scene-referred to display-referred. The rendering colorspace might be an RGB space or an LMS space, or something else entirely.
  • Gamut Mapping - We may want some handling of chromaticities which lie outside of our target output gamut.
  • Whitepoint - We may want to creatively change the whitepoint of our output in order to create warmer or cooler colors in our output image regardless of what display device whitepoint we are outputting to.
  • Rendering Transform - There are many valid approaches for this. ACES uses a per-channel approach where the tonescale curve is applied directly to the RGB input channels, as you were doing in your above experiments. Since I’m working on a chromaticity-preserving display rendering transform, I’ll outline what might go into that.
    • Separate color and grey using some norm.
    • Apply some tonescale to grey to convert from scene-referred to display-referred.
    • Apply some model for surround compensation and flare copmensation.
    • Apply some chroma compression to highlights so that very bright saturated colors are nudged towards the achromatic axis as their luminance increases towards the top of the display gamut volume.
  • Display Gamut Conversion - Convert from our rendering colorspace into our display gamut. If you’re looking for a tool do do arbitrary gamut and whitepoint conversions in Nuke you could check out GamutConvert.nk.
  • InverseEOTF -We need to apply the inverse of the Electro-Optical Transfer Function that the display will apply to the image before emitting light from your display.
  • Clamp - And lastly, maybe we want to clamp the output into a 0-1 range, to simulate what is going to happen when we write the image out into an integer data format.

EOTF

The “DisplayEOTF” gizmo is pretty old technology. It is basically just the Inverse EOTF guts of the Nuke ACES Output Transform implementation I did, ripped out and put in it’s own node. I would recommend that you use the InverseEOTF node in the ElectroOpticalTransferFunction.nk file I posted above instead. It’s a lot simpler.

Nuke Specifics

If you’re using a stock ACES config in Nuke, and your read node’s input colorspace is set to ACES - ACES2065-1 as I see in your screenshot, this is doing a gamut conversion from AP0 to AP1. This makes your “Rendering colorspaces” ACEScg. This is the same as the ACES Output Transforms. The ACEScg → Rec709 matrix is essentially your display gamut conversion. and your DisplayEOTF node is your Inverse EOTF curve.

Hope this big wall of text helps clarify things, or at least brings some new questions that will help in your journey of learning.

2 Likes
61

We most certainly want to add appearance mapping somewhere inside or around those points.

62

Cool! Can you type up a verbose description of what is should be called, where it should go and what exactly it might do, with references?

:slight_smile:

63

You started to put some with the surround compensation and we have all that defined here by @ChrisBrejon via myself: About issues and terminology - #5 by ChrisBrejon

But we need to decide what we want to model and where we put it.

Here is another relevant reference @jedsmith: HPA Tech Retreat 2018: Parametric Appearance Compensation

1 Like
64

Thanks @jedsmith that’s extremely helpful!
Based on your post I’ve chained together the following nodes:

  • gamut compress
  • tonemap (Piecewise Hyperbolic Presets)
  • Gamut convert (ACEScg to Rec709)
  • Inverse EOTF (sRGB display)

alongside that I have your OpenDRT (v0.0.75) which, if I understand correctly does all of the above, plus chroma preserving.

here’s a screenshot of both (OpenDRT is the bottom image):

EDIT: I have updated the images to correct an error in my original settings for the OpenDRT.

image
image1269×789 284 KB

Note how the “warm sunshine yellows” go to what I would describe as a “tangerine orange.” Is this is a necessary consequence of the chroma-preserving nature of the OpenDRT?

Here’s another comparison of the two, first of the Hyperbolic on the left and OpenDRT on the right:

image
image1104×501 118 KB

While the Hyperbolic is not maintaining the colors (the 12 colors become 6), what it is doing is all going to white with increased exposure (compare the last row of both images). The OpenDRT is not going to white at 20 stops. In fact it looks the same at 100 stops as it does at 20. So it appears to never go to white. Perhaps that’s an artifact of this being a CG render, and the same would not happen in a photo? I’ll need to do further tests there…

65

And here is the PPT from that presentation.

1 Like
66

Why is “going to white” important?

What is a “specular highlight”?

67

Hey @Derek, thanks for the further testing.

First, it looks like there’s an error in the setups you picture: The input gamut in the OpenDRT node should be ACEScg not ACES. This error is causing the output to be horribly over-saturated. Maybe you want to edit your post and update the images?

This is a valid creative preference. However, I don’t believe the display rendering transform is the correct place to apply creative preference. If the scene looked “tangerine orange” with your eyes, and the scene looks the same when displayed on a monitor, the display rendering is doing its job. My goal is a faithful representation of the chromaticities represented by the input image. It’s not to say that creative adjustments can not happen, they would just not be part of the display transform. I believe this approach would actually allow more creative freedom because it would allow for varying creative preferences without altering the display rendering transform.

Of course the missing piece here is the LMT, which would apply some sort of creative look. I know it’s difficult to imagine without being able to see something working.

Hope that makes sense, and I hope you’ll update those images with the proper gamut conversion. Curious to see what your comparisons look like properly rendered.

3 Likes
68

Thanks @jedsmith I have updated my post above. Very much appreciate you correcting my error!

69

Hi @Troy_James_Sobotka
by “going to white with increased exposure” I’m referring to what Alex Fry discusses in his famous ACES talk, beginning around minute 7:

I believe you guys are calling that “path to white”?

70

They are sincere questions. What does it mean to you?

  1. “Path to white”? What is it?
  2. “Specular highlights”? What are they?

The reason for these seemingly pedantic questioning is because it forces us collectively to analyze what these things mean.

In terms of light data in an open, radiometric-like domain such as that from a render or a decoded camera massage fest, the terms seem to have extremely slippery definitions.

71

Giving it a stab:

Highlights desaturation process that recovers clipped colour image channel data to produce a pleasing rendering.

Fraction of the radiant power unscattered and reflected from the surface of an object, at an angle opposite but equal to the angle of incidence.

72

Does it “recover”?

Seems to have little to do with a light data buffer?

73

@Troy_James_Sobotka
I appreciate the intent and importance of your question. I’m coming at this from the perspective of an artist, and so if you are looking to have these terms defined in scientific terms (again a totally valid aim), I’m afraid I won’t be much help there. I can say what it means to me as an artist if that would be helpful.

74

So we do not get stuck on display at full emission R, G or B ? Because it reminds us of films ? Or maybe because it is the only way to respect the RGB ratios (or creative/scene intent) at display ? So they don’t get distorted ?

It is also called “bleaching” or “highlights desaturation” ? It is the path that a certain chromaticity would follow towards the achromatic axis as exposure increases ?

Oh my god, I’m terrible at this ! :stuck_out_tongue: I answered your questions by more questions.