ACES background document

Thanks guys. Lots of interesting thoughts here ! I honestly believe we’re making some progress. I’ll try to reply to each point to the best of my abilities. So please bear with me ! :wink:

@alexfry yes, we have thought of the LMT implementation. Thanks for pointing that out ! Issue is that we’d need someone with the proper knowledge to do so (aka a color scientist/TD). And I think lots of small studios/freelancers would be in the same situation, right ? My take is this : would it be fair to say that a Color Management System (CMS) should be able to display properly any values it gives access to, out-of the-box ? I think that’d be a nice improvement. Of course, devil is in details and what do I mean by “display properly” is an open question.

@Alexander_Forsythe thanks for the answer ! This is great ! Let’s for the sake of argument that we agree that mapping from scene to display is a creative choice. What is the best starting point here ? Out-of-the box, no LMT, which option seems more reasonable to you ?

Please remember the scenario where we have hundreds of CG artists from layout to compositing putting colors in their scenes. As a supervisor, which starting point would you rather have for your teams ? Could there be a bucket #3 where we compensate scene values because the display rendering transform is twisting hues ?

@garrett.strudler , cool to have you here ! Don’t apologize. I am the king for making false assumptions ! Yes, I’d have expected my blue neon to go to white earlier and I agree this is debatable. But I disagree on the second argument : about the re-light. On any CG movie, this argument would be true. But not on the renders I provided. For two reasons :

  • In the blue sphere example, there is only one light. So apart of tweaking the global exposure, there is no re-light possible. Maybe by mapping the light with an HDR map that would go from white in the center to blue on the edges, like you suggested ? Not even sure that’d work. Which brings me back to my earlier question : how do we make the life of hundreds of CG artists easier ? If we have to map every light with a gradient (which we barely do on a CG show), does it say something about the DRT ?
  • Same thing with the light sabers. I am using ACEScg primaries directly on the geometries themselves. There is nothing to re-light here. Just maybe a global exposure in my opinion. I don’t even think that texturing the light sabers would work. I have tried to desaturate the lights but then the glow is not colored enough. I really want to assure you that those renders are 100% production compliant, lit exactly how we’d do in a studio.

Again, as an artist, what would be the ideal starting point here :

Look at the cheek, the neck and the t-shirt :

We are all on the same mission here and let me give you a hint of what M. Giorgianni has written on the topic :

Appropriate gamut mapping would then be applied to transform those colors as needed for real outputs. […] It would be expected, then, that some form of gamut mapping will be required in going from the Output CES to any real output device or medium.

I think the key is not to think only about luminance range but also about the gamut volume. We go from one volume to another. Again by M. Giorgianni :

It allows information to be encoded in a way that places no limits on luminance dynamic range or color gamut. Any color that can be seen by a human observer can be represented.

Hello @ruslan.sorokin and welcome to acescentral ! Again, for the sake of argument, let’s say we agree that the mapping is subjective/creative. Did you have a chance to look at this thread where I tried to point at some of the issues we are talking about ? Please have a look at Nick’s gifs and Jed’s videos here. It is important that we agree that per-channel’s path to white is an accident, maybe even an happy one. :wink:

But as an artist, I want control. If I like skews, for an explosion maybe, I 'd rather introduce them as a creative grade or a LMT. Again, have a look the renders and comparisons here : what is the most reasonable starting point ? If I want a blue sphere to look purple under the sunlight, I think it should be a decision/choice, not a default behaviour.

I have also done my homework and asked a dozen of supervisors about a radical change in the Ouput Transforms for ACES 2.0. Mostly from worldwide VFX and Animation studios. I have been given pretty much every time the same answer :

  • A radical change for the ACES 2.0 Output Transforms (such as hue-preserving) will not affect our CG assets.

For three main reasons :

  • Assets are generally lookdeved under a “neutral” transform, which is not the show transform.
  • If a perfect match is needed, the show would just work with the same ACES OCIO config previously used. Just like a software version.
  • Assets are always improved between shows because of software updates and technology changes. No nothing new under the sun. :wink:

As a studio, we are definitely interested by Jed’s experiment and any quality improvement that could be made to the Output Transforms.

Happy to discuss,
Chris

Camera, display and software were engineered. We use color science to solve engineering tasks to satisfy our aesthetic requirements. “Chemical engineers” done the same thing when invented emulsions. So, term “engineered” is a little bit confusing me.

If we talk about changing image contrast. This is not colorimetrical term, right? There is no “natural” transformation. There are only engineering ones. Per-channel is the only simplest one. No more engineered than another.

Film emulsion has 3(4) layers. Each layer has own spectral sensitivity and can be characterized by some transfer function. So, the behavior is pretty much like “per-channel”, no?

Hey guys,

I have thought of a couple more things overnight. :wink:

I always go back to this simple example. What’s happening here ? By increasing the exposure on a sRGB blue primary, the blue channel is arriving first at 1 (or 100% emission) and cannot go beyond. Indeed displays have limitations and do not magically go to white. Then the red channel and green channels catch up creating a hue shift towards purple.

hue_skew_bl

Shall we agree on the following :

  • It is desirable to respect the creative intent when going from scene to display.
  • Per-channel distorts the RGB ratios of the scene and therefore does not respect the creative intent.
  • If such a hue shift was desirable, it should be a creative decision and not a default behavior.

I think it would be a step forward if we agreed on these three statements.

I have said earlier in this post that adding 7-8% of green would be a compensation for this behavior. I think it is worth nothing that by doing so, we are just changing the skew direction and not necessarily fixing the issue.

hue_skew_fix_bl

The epiphany I have had so far with this VWG is that the path to white must be engineered on a digital display (it was also engineered for film, agreed, but I would not remotely compare per-channel with film emulsion). We always come back to this fundamental question :

What should happen to values that are even more intense than 100% emission ? Just leave them stuck at the maximal value ?

And since we are as well having this great debate about film vs eye, I thought of another silly experiment :

  • Put on a blue shirt, go outside to get hit by sunlight. Does the blue shirt get purple ? Do you expect it to get purple ?
  • Go back home, put on a red shirt, go outside to get hit by sunlight. Does the red shirt get orange ? Do you expect it to get orange ?

Finally, per-channel is indeed engineered (nice catch @ruslan.sorokin ) but shall we agree that it is the responsibility of this VWG to at least investigate and discuss more elaborate transforms ? Since respecting the creative intent is our goal here ?

A side note : I would not literally spend hours writing these posts and providing these examples if I didn’t care about ACES. I think this VWG has some of the best people in the industry which is why I have high hopes/expectations for the outcome.

Update : I forgot to share with you a beautiful answer from one the lookdev supervisors I contacted.

[…] the calibration of an asset should be completely independent of the quality of the Output transform, of the tone mapping and in the broad sense of the look (CDL, LUT, etc …) applied to the images.

Chris

3 Likes

Hey guys,

Jed Smith has done another brilliant video that I wanted to share with you. If every single person from the VWG could have a look at it, I think it’d be a good thing. :wink: It is only 40 frames long and it shows us the path to white on ACEScg samples through the current Rec.709 ACES Output Transform. And it is awesome.

During the TAC meeting, it has been said :

There’s two implementations here […] Whether you should facilitate the maximally saturated output of a given color, whether you should enforce a desaturation towards white in order that you get these kind of nice hue property […]. And those two things directly are competing because one desaturates, the other one requests you not to […].

Although I do not disagree with this statement, I think it can be misleading. I think there is only one acceptable solution : not collapsing values at display on their path to white. I would rephrase it this way :

  • Facilitating maximally saturated output of a given color (which is the supposed current behavior) is actually more of a collapse of the values on display gamut boundaries.
  • Enforcing a desaturation towards white is acknowledging the fact that displays have limitations and scene radiometric values should be compressed. We are not enforcing anything really.

I have made some screenshots of the video to explain as best as I could.

Frame 1 : we start with low exposure values. So far, so good.

Frame 14 : red values start to collapse on the display limitations. Because a Rec.709 display monitor has limitations.

Frame 19 : blue and purple values collapse on the display limitation. Pay attention to the hue deviations/twists.

Frame 21 : green values collapse on the display limitations.

Frame 26 : The per-channel “accident” kicks in. Values “bounce back” from display limitations in order to go to white.

Frame 32 : Remember the original samples : they were a sweep of the whole spectrum. On their path to white, they collapsed into 6 values (aka the notorious 6).

You have the right to like this behavior. But I really don’t think this should be the default output transform. I think we can and should do better. We are trying to reproduce faithfully a scene by taking in account the display limitations, right ? And in order to do so, we need to compress values, which per-channel doesn’t do (well enough/at all).

And how we compress these values nicely/smartly/elegantly is actually a point of debate. Exactly like Lars Borg said in meeting#5 :

The effectiveness of this method will mostly rely on the quality of the gamut compression.

This is why the subjective/creative argument about per-channel does not work for me. I really don’t think that collapsing values is about taste. And this is what Jed is trying to address with his naive DRT.

I am more than happy to hear suggestions/ideas/comments on how things could be handled differently. But I don’t thing we can demonstrate things more clearer than that.

  • If values don’t go to white, where should they go ?
  • Per-channel is an (engineered) accident that does not really compress anything.
  • Per-channel and film emulsion have not much in common.
  • Per-channel does not facilitate maximally saturated output of a given color, it just collapses and twists them in a way that you may like, but that should not be default behaviour.

Finally, I would like to quote Rod Bogart from the TAC meeting :

the efforts that we’ve been going though, over the years, have led to this, that we couldn’t do this first. […] And now that we’re here, it’s time.

Now is the time do something awesome. :wink: Here is the Nuke script if you want to play with it.
ACES_Rec.709_Swatches_Exposure_Visualization.nk (146.9 KB)

Update : Jed has done the same video with his DRT so we can compare. Thanks Jed ! :wink:

And for completeness, two final images (thanks again Jed !).
A pure AP1 hue sweep rendered through the ACES Rec.709 output transform :

This image is the same but the source with 90% of the AP1 gamut volume as input.

Regards,
Chris

1 Like

I’m having a lot of fun making plots and visualizations lately. Here is another one that maybe shows more clearly what is happening.


(Right-click and loop and watch it a few times… it’s pretty mesmerizing).

What is happening there?

  • We start with six (implicitly AP1) RGB color swatches on the primary and secondary colors (Red, Green, Blue, Cyan, Magenta, Yellow)
  • Each swatch varies in intensity from 0 to 1
  • We do a hexagonal desaturation to “place” the position of the color swatches relative to the gamut boundary. Saturation of 1 is at the gamut boundary. Saturation of 0.5 is 50% towards the achromatic axis (max(r,g,b)). This visualization shows the colors at 10% inward from the gamut boundary.
  • We do a rotation of hue in HSV, from 0 to 60 degrees. The animation is 60 frames, each frame is 1 degree of rotation. The first frame is 0 degrees rotation, where all swatches align to the “notorious six” (primaries and secondaries).
  • We expose up by some number of stops.
  • We run these colors through the ACES Rec.709 Output Transform. (Note that we output display linear, in order to plot the colors as they would appear on the display, effectively bypassing the inverse EOTF + EOTF conversion).

Just thought I’d share this too in case it helps anyone visualize what is happening more effectively.

As usual, here’s the Nuke setup that generated that plot if anyone wants to play with it.
plot_aces_rec709_output_transform_rgb_six_rotate.nk (133.3 KB)

And here is a few more stills that show what is being clipped beyond the display gamut cube.

At 0.9 times the AP1 gamut boundary:



And the same with 15 degrees hue rotation:

And at 1.0 times the AP1 gamut boundary:



And the same wtih 15 degrees hue rotation:

3 Likes

I was doing a bit of thinking this morning about one of the conflicting design requirement that @KevinJW mentioned in the TAC meeting:

facilitate the maximally saturated output vs enforce a desaturation towards white

To better understand the behavior of each approach, and therefore the pros and cons, I put together … you guessed it: some more plots!

Here’s the setup. We have a 64 frame animation, featuring two exposure sweeps from 0 to +8 stops, of a 6 patch hue swatch of primary and secondary colors, and a second exposure sweep of the same image with a 15 degree hue rotation. For clarity, each hue swatch is a linear ramp from 0 to full color in x and a linear ramp from full saturation to 80% saturation in y.

There are 4 videos with different display rendering transforms.


First we have a pure inverse eotf + clamp. Note the typical behavior of saturated colors converging to the primary and secondary colors as the values clip in overexposure, and then converging to white.


Next we have the same setup rendered through the aces rec.709 output transform. Note the same behavior as the first video in colors between the primaries and secondaries as they converge to primaries and secondaries in overexposure. (Of course the behavior is a bit more nuanced and nonlinear in the aces transform). (Note: I’ve disable the acescg to rec.709 display colorimetry conversion here so that this aspect doesn’t confuse us.)


Next we have the same image rendered through the NaiveDisplayTransform_v03.nk available in this post. (Note: I’ve disable the acescg to rec.709 display colorimetry conversion here as well)


And then the same with “path to white” disabled.

As usual, here’s the nuke script that generated these plots. overexposure_comparisons.nk (24.4 KB)

A couple of observations.

  • The range of brightness expressed by the “hue preserving no path to white” display rendering is very limited. For an SDR display rendering I think this is a huge downside.
  • The main argument I’ve seen mentioned for having “no path to white” is HDR. It has been mentioned that very bright and very saturated colors are often desired in HDR. It’s difficult for me to know without being able to test on any HDR display. However my intuition tells me we would still want a “path to white” above a certain threshold, it’s just that this threshold would be different between HDR and SDR. The question is, how different? And could it be handled purely by the difference in rendering transform?

One of the biggest arguments for me in favor of a chromaticity preserving display rendering is more consistent rendering of color in highlights between SDR and HDR. I’m guessing here without an HDR display, but I think the hue twists inherent in a per-channel display rendering approach would be very difficult to undo when using an HDR rendering.

The only person I’ve seen mention this is @daniele, on the colour-science slack, so I thought I’d bring it up here.

4:18 PM Saturday Jan 30th
Daniele Siragusano I am not saying HDR and SDR should be identical. But a yellow candle in SDR should not be orange in HDR I guess
At least I know many colourists which are confused when that happens.

2 Likes

Here are some demos of “maximally saturated” with respect to the display gamut volume, without any chromaticity compression of any sort. The values are engineered to remain “fixed” at the limits of the display gamut boundary.

The following images use a generic S shaped aesthetic tonality curve. It should be noted that when using a chromaticity preserving rendering approach, the shape and form of the curve is of much lower consequence than than range it attempts to compress to the display.

Note that this is for over exposure, and the same problems arise for under exposure.

The EV presented here are relative to the source image’s default encoding.

-2 EV:

-1 EV:

+/- 0 EV:

+1 EV:

+2 EV:

+3 EV:

+4 EV:

Here is a representation of the values that are out of gamut at a mere +1 EV:

Great videos, they showcase well the orthogonality between no/path to white.

It is not necessarily HDR related, people have for example complained about how difficult it is to get saturated yellows with ACES SDR transforms for years. I would rephrase that more as some people wants/need to get very close or even touch the volume corner. With a transform that aggressively desaturates everything in the highlights, it is super hard, it is already with ACES OTs.

The core question to me is how we enable both?

One thing worth trying is a no-path to white chromaticity preserving transform with a path to white LMT that is effectively a hue-keyer allowing you to selectively control which hues are allowed to hit the full extent of the volume and which ones are not. I have done that at times :slight_smile:

Cheers,

Thomas

Conceptually an LMT is a transform which should be the same for all viewing condition (a scene-referred edit). The path to white might be different for different viewing conditions. It might be difficult to get a balance here if you do a binary split.

2 Likes

Yes the current LMTs are scene-referred but I’m also thinking about output-referred ones here, before the final display encoding. So you end-up with a single RT and a stack of output-referred LMTs that can be used to tweak the appearance on different displays instead of forcing that from a under the stack which ultimately is impossible to do with full control.

2 Likes

Without clear design requirements, we won’t get nowhere and we’'ll end up chasing our tails/running in circles.

Then we’ll talk solutions.

Chris

It strikes me that gamut compression concerns are adjacent to perceptual concerns. Do you see a way to pull out perceptual facets based on the internal “intentions” of the models? EG: Can a perceptual facet be negotiated at the very last stage in reconciliation with the display?

Something that was not noted in the issues on Dropbox Paper is the hard time to get saturated yellows with the current ACES DRT which was reported quite often.

Here is a video showing the current volume filled as exposure increases (sorry for the low sampling quality, I don’t have Jed’s budget :D)

Note that the distortions are caused mostly by the 3D LUT and interpolation.

And for good measure, Jed’s Naive DRT with default settings:

Note that the bias can be adjusted to get closer to the boundaries if required.

It is of course better viewed directly in Nuke: DRT Cubes · GitHub

Cheers,

Thomas

It becomes tricky from an architectural and user experience standpoint if you start to introduce a stack of transforms, which together form the output transform.

I think what we are doing (scattered around this forum) is collecting parameter for a possible set of viewing transform. I think the real work is to agree on a balance of those parameters for a default viewing transform, knowing that many parameters (or even all) can only be set with a large compromise. A small group could agree on such tuning of those parameters, but as soon as you widen the discussion group it becomes unmanageable. Hence the ability to swap output transforms could save us here.

2 Likes

We should never forget that the net experience of a pipeline is a complex interplay of working space definition and output transform.
I personally do not put so much weight into the display-referred state, because we are here in the domain of image creation. So a skilled person can tweak things. Some edge cases can be loaded onto the shoulders of the content creators, as long as the pipeline is elastic. This can lead into a much simpler design.

…And I know that you would disagree :-).

Sure but that is why you have higher level abstractions that make it “easy” to users. For example people do not compose the RRT with an ODT when they pick up their ACES DRT, they just choose one and it assembles the building blocks for them.

Yes, although I see it more like a set of lego bricks we put together, hence the output-referred LMTs. I’m not saying we should do that though, I’m shaking the idea bin and see if anything good comes down!

Unrelated, Naive DRT with 0.005 bias:

Cheers,

Thomas

1 Like

I’m just getting my feet wet in Nuke, so thank you for the samples. This is one of the concerns I’ve been thinking about lately with the “naive” DRT, so your timing is quite good! There are corners (or quite more) of a display’s gamut that you could simply never get to with the naive DRT. If this is already a current criticism then that definitely should be taken under advisement.
(side note: while I am often critical of the “naive” DRT as a RRT solution, I think it could make a good LMT for those wanting a nice “path to white” if the RRT doesn’t do this).

As a “universal” workflow/pipeline (and archive) I personally feel strongly that we shouldn’t restrict access to color volume/gamut as much as is feasible. I know @ChrisBrejon compiled a list of suggested/proposed requirements in another thread (forgot which one) and in addition to the Dropbox paper, but I think this should also at least be considered. However, while not necessarily directly conflicting, it certainly presents challenges with regards to “no gamut clipping” if we allow values right up to the boundaries.

@garrett.strudler Design requirements are available on the paper dropbox, and eventually in this separated thread. I had written some of them in this thread as well but I think it is better to have a separate thread. Please join !

Just to be clear, I don’t think anyone is proposing to do that. All values will still be available in the master/archive as always. But I do think the display limitations need to be accounted for : when a value reaches 100% emission of the display, how can we keep the RGB ratio intact and respect therefore the creative intent ?

Thanks,
Chris