ACES background document

Hey guys,

Jed Smith has done another brilliant video that I wanted to share with you. If every single person from the VWG could have a look at it, I think it’d be a good thing. :wink: It is only 40 frames long and it shows us the path to white on ACEScg samples through the current Rec.709 ACES Output Transform. And it is awesome.

During the TAC meeting, it has been said :

There’s two implementations here […] Whether you should facilitate the maximally saturated output of a given color, whether you should enforce a desaturation towards white in order that you get these kind of nice hue property […]. And those two things directly are competing because one desaturates, the other one requests you not to […].

Although I do not disagree with this statement, I think it can be misleading. I think there is only one acceptable solution : not collapsing values at display on their path to white. I would rephrase it this way :

  • Facilitating maximally saturated output of a given color (which is the supposed current behavior) is actually more of a collapse of the values on display gamut boundaries.
  • Enforcing a desaturation towards white is acknowledging the fact that displays have limitations and scene radiometric values should be compressed. We are not enforcing anything really.

I have made some screenshots of the video to explain as best as I could.

Frame 1 : we start with low exposure values. So far, so good.

Frame 14 : red values start to collapse on the display limitations. Because a Rec.709 display monitor has limitations.

Frame 19 : blue and purple values collapse on the display limitation. Pay attention to the hue deviations/twists.

Frame 21 : green values collapse on the display limitations.

Frame 26 : The per-channel “accident” kicks in. Values “bounce back” from display limitations in order to go to white.

Frame 32 : Remember the original samples : they were a sweep of the whole spectrum. On their path to white, they collapsed into 6 values (aka the notorious 6).

You have the right to like this behavior. But I really don’t think this should be the default output transform. I think we can and should do better. We are trying to reproduce faithfully a scene by taking in account the display limitations, right ? And in order to do so, we need to compress values, which per-channel doesn’t do (well enough/at all).

And how we compress these values nicely/smartly/elegantly is actually a point of debate. Exactly like Lars Borg said in meeting#5 :

The effectiveness of this method will mostly rely on the quality of the gamut compression.

This is why the subjective/creative argument about per-channel does not work for me. I really don’t think that collapsing values is about taste. And this is what Jed is trying to address with his naive DRT.

I am more than happy to hear suggestions/ideas/comments on how things could be handled differently. But I don’t thing we can demonstrate things more clearer than that.

  • If values don’t go to white, where should they go ?
  • Per-channel is an (engineered) accident that does not really compress anything.
  • Per-channel and film emulsion have not much in common.
  • Per-channel does not facilitate maximally saturated output of a given color, it just collapses and twists them in a way that you may like, but that should not be default behaviour.

Finally, I would like to quote Rod Bogart from the TAC meeting :

the efforts that we’ve been going though, over the years, have led to this, that we couldn’t do this first. […] And now that we’re here, it’s time.

Now is the time do something awesome. :wink: Here is the Nuke script if you want to play with it.
ACES_Rec.709_Swatches_Exposure_Visualization.nk (146.9 KB)

Update : Jed has done the same video with his DRT so we can compare. Thanks Jed ! :wink:

And for completeness, two final images (thanks again Jed !).
A pure AP1 hue sweep rendered through the ACES Rec.709 output transform :

This image is the same but the source with 90% of the AP1 gamut volume as input.

Regards,
Chris

1 Like

I’m having a lot of fun making plots and visualizations lately. Here is another one that maybe shows more clearly what is happening.


(Right-click and loop and watch it a few times… it’s pretty mesmerizing).

What is happening there?

  • We start with six (implicitly AP1) RGB color swatches on the primary and secondary colors (Red, Green, Blue, Cyan, Magenta, Yellow)
  • Each swatch varies in intensity from 0 to 1
  • We do a hexagonal desaturation to “place” the position of the color swatches relative to the gamut boundary. Saturation of 1 is at the gamut boundary. Saturation of 0.5 is 50% towards the achromatic axis (max(r,g,b)). This visualization shows the colors at 10% inward from the gamut boundary.
  • We do a rotation of hue in HSV, from 0 to 60 degrees. The animation is 60 frames, each frame is 1 degree of rotation. The first frame is 0 degrees rotation, where all swatches align to the “notorious six” (primaries and secondaries).
  • We expose up by some number of stops.
  • We run these colors through the ACES Rec.709 Output Transform. (Note that we output display linear, in order to plot the colors as they would appear on the display, effectively bypassing the inverse EOTF + EOTF conversion).

Just thought I’d share this too in case it helps anyone visualize what is happening more effectively.

As usual, here’s the Nuke setup that generated that plot if anyone wants to play with it.
plot_aces_rec709_output_transform_rgb_six_rotate.nk (133.3 KB)

And here is a few more stills that show what is being clipped beyond the display gamut cube.

At 0.9 times the AP1 gamut boundary:



And the same with 15 degrees hue rotation:

And at 1.0 times the AP1 gamut boundary:



And the same wtih 15 degrees hue rotation:

3 Likes

I was doing a bit of thinking this morning about one of the conflicting design requirement that @KevinJW mentioned in the TAC meeting:

facilitate the maximally saturated output vs enforce a desaturation towards white

To better understand the behavior of each approach, and therefore the pros and cons, I put together … you guessed it: some more plots!

Here’s the setup. We have a 64 frame animation, featuring two exposure sweeps from 0 to +8 stops, of a 6 patch hue swatch of primary and secondary colors, and a second exposure sweep of the same image with a 15 degree hue rotation. For clarity, each hue swatch is a linear ramp from 0 to full color in x and a linear ramp from full saturation to 80% saturation in y.

There are 4 videos with different display rendering transforms.


First we have a pure inverse eotf + clamp. Note the typical behavior of saturated colors converging to the primary and secondary colors as the values clip in overexposure, and then converging to white.


Next we have the same setup rendered through the aces rec.709 output transform. Note the same behavior as the first video in colors between the primaries and secondaries as they converge to primaries and secondaries in overexposure. (Of course the behavior is a bit more nuanced and nonlinear in the aces transform). (Note: I’ve disable the acescg to rec.709 display colorimetry conversion here so that this aspect doesn’t confuse us.)


Next we have the same image rendered through the NaiveDisplayTransform_v03.nk available in this post. (Note: I’ve disable the acescg to rec.709 display colorimetry conversion here as well)


And then the same with “path to white” disabled.

As usual, here’s the nuke script that generated these plots. overexposure_comparisons.nk (24.4 KB)

A couple of observations.

  • The range of brightness expressed by the “hue preserving no path to white” display rendering is very limited. For an SDR display rendering I think this is a huge downside.
  • The main argument I’ve seen mentioned for having “no path to white” is HDR. It has been mentioned that very bright and very saturated colors are often desired in HDR. It’s difficult for me to know without being able to test on any HDR display. However my intuition tells me we would still want a “path to white” above a certain threshold, it’s just that this threshold would be different between HDR and SDR. The question is, how different? And could it be handled purely by the difference in rendering transform?

One of the biggest arguments for me in favor of a chromaticity preserving display rendering is more consistent rendering of color in highlights between SDR and HDR. I’m guessing here without an HDR display, but I think the hue twists inherent in a per-channel display rendering approach would be very difficult to undo when using an HDR rendering.

The only person I’ve seen mention this is @daniele, on the colour-science slack, so I thought I’d bring it up here.

4:18 PM Saturday Jan 30th
Daniele Siragusano I am not saying HDR and SDR should be identical. But a yellow candle in SDR should not be orange in HDR I guess
At least I know many colourists which are confused when that happens.

2 Likes

Here are some demos of “maximally saturated” with respect to the display gamut volume, without any chromaticity compression of any sort. The values are engineered to remain “fixed” at the limits of the display gamut boundary.

The following images use a generic S shaped aesthetic tonality curve. It should be noted that when using a chromaticity preserving rendering approach, the shape and form of the curve is of much lower consequence than than range it attempts to compress to the display.

Note that this is for over exposure, and the same problems arise for under exposure.

The EV presented here are relative to the source image’s default encoding.

-2 EV:

-1 EV:

+/- 0 EV:

+1 EV:

+2 EV:

+3 EV:

+4 EV:

Here is a representation of the values that are out of gamut at a mere +1 EV:

Great videos, they showcase well the orthogonality between no/path to white.

It is not necessarily HDR related, people have for example complained about how difficult it is to get saturated yellows with ACES SDR transforms for years. I would rephrase that more as some people wants/need to get very close or even touch the volume corner. With a transform that aggressively desaturates everything in the highlights, it is super hard, it is already with ACES OTs.

The core question to me is how we enable both?

One thing worth trying is a no-path to white chromaticity preserving transform with a path to white LMT that is effectively a hue-keyer allowing you to selectively control which hues are allowed to hit the full extent of the volume and which ones are not. I have done that at times :slight_smile:

Cheers,

Thomas

Conceptually an LMT is a transform which should be the same for all viewing condition (a scene-referred edit). The path to white might be different for different viewing conditions. It might be difficult to get a balance here if you do a binary split.

2 Likes

Yes the current LMTs are scene-referred but I’m also thinking about output-referred ones here, before the final display encoding. So you end-up with a single RT and a stack of output-referred LMTs that can be used to tweak the appearance on different displays instead of forcing that from a under the stack which ultimately is impossible to do with full control.

2 Likes

Without clear design requirements, we won’t get nowhere and we’'ll end up chasing our tails/running in circles.

Then we’ll talk solutions.

Chris

It strikes me that gamut compression concerns are adjacent to perceptual concerns. Do you see a way to pull out perceptual facets based on the internal “intentions” of the models? EG: Can a perceptual facet be negotiated at the very last stage in reconciliation with the display?

Something that was not noted in the issues on Dropbox Paper is the hard time to get saturated yellows with the current ACES DRT which was reported quite often.

Here is a video showing the current volume filled as exposure increases (sorry for the low sampling quality, I don’t have Jed’s budget :D)

Note that the distortions are caused mostly by the 3D LUT and interpolation.

And for good measure, Jed’s Naive DRT with default settings:

Note that the bias can be adjusted to get closer to the boundaries if required.

It is of course better viewed directly in Nuke: DRT Cubes · GitHub

Cheers,

Thomas

It becomes tricky from an architectural and user experience standpoint if you start to introduce a stack of transforms, which together form the output transform.

I think what we are doing (scattered around this forum) is collecting parameter for a possible set of viewing transform. I think the real work is to agree on a balance of those parameters for a default viewing transform, knowing that many parameters (or even all) can only be set with a large compromise. A small group could agree on such tuning of those parameters, but as soon as you widen the discussion group it becomes unmanageable. Hence the ability to swap output transforms could save us here.

2 Likes

We should never forget that the net experience of a pipeline is a complex interplay of working space definition and output transform.
I personally do not put so much weight into the display-referred state, because we are here in the domain of image creation. So a skilled person can tweak things. Some edge cases can be loaded onto the shoulders of the content creators, as long as the pipeline is elastic. This can lead into a much simpler design.

…And I know that you would disagree :-).

Sure but that is why you have higher level abstractions that make it “easy” to users. For example people do not compose the RRT with an ODT when they pick up their ACES DRT, they just choose one and it assembles the building blocks for them.

Yes, although I see it more like a set of lego bricks we put together, hence the output-referred LMTs. I’m not saying we should do that though, I’m shaking the idea bin and see if anything good comes down!

Unrelated, Naive DRT with 0.005 bias:

Cheers,

Thomas

1 Like

I’m just getting my feet wet in Nuke, so thank you for the samples. This is one of the concerns I’ve been thinking about lately with the “naive” DRT, so your timing is quite good! There are corners (or quite more) of a display’s gamut that you could simply never get to with the naive DRT. If this is already a current criticism then that definitely should be taken under advisement.
(side note: while I am often critical of the “naive” DRT as a RRT solution, I think it could make a good LMT for those wanting a nice “path to white” if the RRT doesn’t do this).

As a “universal” workflow/pipeline (and archive) I personally feel strongly that we shouldn’t restrict access to color volume/gamut as much as is feasible. I know @ChrisBrejon compiled a list of suggested/proposed requirements in another thread (forgot which one) and in addition to the Dropbox paper, but I think this should also at least be considered. However, while not necessarily directly conflicting, it certainly presents challenges with regards to “no gamut clipping” if we allow values right up to the boundaries.

@garrett.strudler Design requirements are available on the paper dropbox, and eventually in this separated thread. I had written some of them in this thread as well but I think it is better to have a separate thread. Please join !

Just to be clear, I don’t think anyone is proposing to do that. All values will still be available in the master/archive as always. But I do think the display limitations need to be accounted for : when a value reaches 100% emission of the display, how can we keep the RGB ratio intact and respect therefore the creative intent ?

Thanks,
Chris

How did you get this so accurate with Rec709 ?

Sorry. I do not understand your question. Could you explain further ?

Thanks !

I meant this post; ACES background document - #6 by Alexander_Forsythe both images are the same, how did you achieve this ?

They aren’t fully the same :slight_smile: but I think there isn’t much more to it than what Christophe explains.

So the ACES version uses the same rgb values input as surface colors as the other but because rendering in ACES means that colorvalues are considered ACEScg (AP1) instead of Rec.709 (unless the sofware is setup to manage otherwise). They weren’t converted from Rec.709 to AP1 before rendering.

Christophe also has a blog covering CG with ACES which I found to be a great resource.

So, these are like two complete different setups.

  • One has ACEScg as a working space and so I use ACEScg primaries in the cornell box and display them with the ACES (Rec.709) Output Transform.
  • The other has “sRGB/BT.709 - linear” as a working space. So I use sRGB/BT.709 primaries for the Cornell Box and displays them with a BT.1886 EOTF.

The idea was indeed to show how close they were visually and see if it would generate any kind of discussion or investigation. Could be interesting to check these renders with the three candidates possibly ?

Chris