ACES 2.0 Seeing a few issues

[jmgilbert (Jean-Michel Gilbert)]

Alright. It took me more time than anticipated since I did a major cleanup of the DCTL after coming back from vacation, integrated all the Github code up to SHA 26ab4fd which I’m guessing is the same version that shipped with Resolve. In that DCTL, I also added the v39 chroma compression and the v39 iterative gamut compressor. Finally, I added a soft clamping to AP1 based on a relaxed version of the reference gamut compression LMT. All those extras are optional and can be enabled separately for tests.

Here is the full DCTL file. Be warned that it’s huge and that makes modifying it harder if you are inclined to do so. Code that is not part of ACES 2 is delimited with markers of the style : // ---- v39 extras ----

With the options that are implemented, you should be able to build almost all D65 output transforms except DCDM which require XYZ-E encoding primaries which I haven’t bothered with. D60 sim isn’t implemented since I haven’t added the required 3x3 matrices with D60 white.

When I have time, maybe I’ll try to figure out why the release version of ACES 2 does what it does and why it creates polarity flips in @Troy_James_Sobotka words.

ACES2_resolve_version_26ab4fd.dctl (79.2 KB)

[chuckyboilo (Charles Boileau)]

Thank you @Troy_James_Sobotka for the explanation.

[ChrisBrejon (Christophe Brejon)]

I have been thinking a bit about this thread and I wanted to share some ideas if that´s okay.

Please take them as a bit of exploration rather than a proper solution.

I think that the OT VWG proved that it is difficult to satisfy everybody when it comes to Output Transforms/Picture Formations. So the idea for ACES 2.0 was to provide a more “neutral” starting point, so everybody could craft more easily their images according to their taste/needs.

But to me this implies an important step in the workflow: the LMTs (or looks by OCIO terminology). And even if it is interesting to compare several Picture Formations “out-of-the-box”, I would argue that it does not tell the full story.

So, having that in mind, I decided to develop a look for ACES 2.0 and I will share here the results.

My first step was to look at an ACEScg sweep where each square represents an increment of one stop.
I am putting here ACES 1.X for comparison:

Most of you have already seen this image with the “infamous” notorious 6.

Now here is the same sweep with ACES 2.0:

At first glance, this looks like an improvement since the “notorious 6” are gone. But as always, devil is in the details. Two things jump out to me: the rates of change and the purity (or chroma) compression seem “off”. It is especially noticeable in the yellows and greens, but this might be because of the requirement to “reach out the corners” of the display gamut.

Here is the same sweep with my LMT (“CB-101”):

Again, I don´t think this LMT should be perceived as some kind of solution for anything but more of an attempt to develop a “pleasing/natural” look under the picture formation.

So, if we take our sunset example, here are the results. First ACES 1.X with the high contrast and shift towards yellow:

Then ACES 2.0 with no shift towards yellow (in the sky region) but the polarity flip (around the sun):

And finally, ACES 2.0 with my LMT where I ty to fix the polarity flip (I don´t think my attempt is 100% successful) and I reintroduced a slight “hue path bending” to avoid the salmon sky of the previous image:

I tested my LMT under a bunch of images and I think it helps in some areas. But I cannot help to think that in some aspects I was fighting the picture formation. For instance, I was not able to fix the blue artefacts shared by Charles in his second example. It would be nice to think of a potential LMT fix for this.

Here are some more examples:
First ACES 1.X with the sRGB spheres and their hue skews:

Then ACES 2.0 with the polarity flip (I think that if you pay real attention, you may notice a dark ring around the specular highlights of the spheres):

And then ACES 2.0 with my LMT (which is far from perfect if you look carefully at the purity compression in some areas):

Here are some more examples:
First ACES 1.X with the bt.2020 spheres and the notorious 6:

Then ACES 2.0 with the “chromaticity-linear approach” but with some loss of tonality in several areas such as reds and blues (notice the harsh line in the bottom row of the blue sphere):

And then ACES 2.0 with my LMT (I tried to improve the overall smoothness but actually made it worse in the blues and yellow purity compression is too high):

I really like these simple examples because this is where it is easier to spot where/how the Picture Formation falls apart or not. Finally, I will share some examples of gradient (using volumetric lights) where I was not able to achieve the desired result.

This is an ACEScg red primary light with ACES 1.X:

This is an ACEScg red primary light with ACES 2.0 (notice the ring breaking the tonality):

This is an ACEScg red primary light with ACES 2.0 and my LMT (I kinda improved the smoothness here but I created this massive white flat area):

This is the “visual target” I was trying to achieve (JP2499):

I have very similar results with an ACESg blue primary light (following the same order):

Apologies for the long post… I will just wrap up with some final thoughts:

• My approach is “limited” in the sense that I did not check any of this footage in HDR and I used a particular set of tools to generate the LMT. As a next step, I will try to use a completely toolset and see if I can improve my LMT. I will report back about this next attempt.
• Personally I would not recommend to use ACES 2.0 without a LMT. So the polarity flips and tonality disruptions observed in some of the examples above are improved .
• I think it would be nice to have a couple of LMTs available in some repo (or config) to help the artists in some of the areas shown above.
• Also, I would recommend that these LMTs re-introduce some “engineered hue path bending” so fires do not look salmon and avoid abney effect with blues for example. Just like the Alex Fry´s presentation from Electronic Arts.

About this last point, I will spare you all the details but I learned the hard way on the movie I am currently working on that “neutral does not look natural nor pleasing”.

Here are the final examples regarding that matter:
ACES 1.X:

ACES 2.0:

ACES 2.0 with my LMT:

I think a good way to think about this is using the Planckian locus/Kelvin temperatures to generate some fire simulations and making sure they do not come as “salmon pink” (I think Liam´s example above uses this already). It worries me that FX artists have to compensate the Picture Formation behavior in their scenes (e.g. the stimuli ain´t the image).

Thanks !

[nick (Nick Shaw)]

@ChrisBrejon

It worries me that FX artists have to compensate the Picture Formation behavior in their scenes

I would perhaps not think of it as compensating. I would say that, as (the other) @alexfry refers to in his presentation, you are “putting it in the data” so it comes out the same in HDR and SDR, rather than leaning into skews in the DRT for the look, which will vary between different targets.

But you make some good points. I think in time people will develop LMTs which give them a preferred rendering under ACES 2.0. There was talk of some being provided by The Academy. But there was not anything ready to ship with the 2.0 release.

Many of the issues discussed may well be related to the need to hit the corners. So if that is not required for a particular project (or scene/shot) an LMT which rounds off the corners may well mitigate those issues.

[ChrisBrejon (Christophe Brejon)]

Thanks Nick.

I did watch the Siggraph presentation from “our” Alex Fry (it is way too short by the way and the topic would have deserved a one-hour presentation at least) and the part about the hue skews is one I disagree with.

I don’t have any technical nor scientific knowledge to back me up though. Only my experience and my eyes that make me think that the workflow described for “Transformers one” is not ideal.

I spent two years on a project using a chromaticity-linear Picture Formation and it is actually stunning how much we need some kind of “hue skews” to make an image look pleasing/natural or dare I say “filmic”. And I think this is why “per-channel” has been so popular for the past 20 years (and still is with AgX and JP2499), because with very little engineering, we would get “colour crosstalk” and “pleasing” images.

The most obvious example are fires and explosions, but actually “hue skews” are needed for pretty much everything (skintones, skies, vegetation…). And so, rather than modifying the data, I personally think that a carefully engineered “hue path bending” (that would behave similarly in SDR and HDR) is a better approach.

This is why I proposed a test where we use Kelvin Temperatures (and not “custom” RGB values) to evaluate the Picture Formation. We could even check with a Physical Sky or some HDRIs now that I think about it.

Chris

PS: One the project I am referring to (pookoo), after two years of production, I developed a “look” for the back-end departments where I re-introduced some hue skews. It helped us with fire of course, but it made our little furry creatures and environment so much more appealing. I really learned my lesson on that one.

[daniele (Daniele Siragusano)]

To author the skews to your liking via an LMT I think it helps if no further skews are introduced by the DRT. Otherwise you are constantly fighting the DRT.
But it makes an LMT mandatory and the most important part in the chain, from a creative standpoint.

[Troy_James_Sobotka (troy_s)]

@daniele

To author the skews to your liking via an LMT I think it helps if no further skews are introduced by the DRT. Otherwise you are constantly fighting the DRT.

This is a solid design goal in theory, the problems that emerge though are subtle:

1. If we do not have a prevailing understanding of how excitation purity must attenuate in a pictorial depiction, we are saddled with an ill posed problem. All evidence points to a variable response that emerges from the surrounding gradients.
2. Given the ill posed nature of the problem surface, we can end up with picture formation algorithms that make critically erroneous assumptions. Examples would be CAM based approaches, or chromaticity linear attenuations, etc.

All decisions are going to suffer from ill posed problems here, making the ability of the author to “get out from” more or less impossible.

If we don’t know what we are doing, whatever it is that we are doing, is wrong.

[ChrisBrejon (Christophe Brejon)]

@daniele

But it makes an LMT mandatory and the most important part in the chain, from a creative standpoint.

Thanks. I know you told me this several times and I should have listened closer. But lesson learned.

Actually, if I understood correctly, ACES 2.0 has a slightly curved path rather than a straight line. And so this is maybe why I had a bit the sensation of fighting it. But my point still stands: the neutral look does not necessarily qualify as “pleasing” and maybe we should evangelize a bit more on that aspect

To me this topic brings back the question to our users (who are they ?) and what do they expect from ACES. Because I feel like the requirement “looking good out-of-the-box” has been left a bit aside but I wonder how often does an ACES project need a perfect full inversion or to reach the display gamut corners ?

@Troy_James_Sobotka

All decisions are going to suffer from ill posed problems here, making the ability of the author to “get out from” more or less impossible.

I kept digging with the new toolset I was mentioning and I was able to get almost where I wanted to be. Except the blues, which is still a stopper for me. Nonetheless, here are my latest results comparing ACES 2 and ACES 2 with this new iteration of “CB-101” LMT.

I agree with you troy that this LMT does not solve anything. It is just another attempt…

Chris

[Troy_James_Sobotka (troy_s)]

@ChrisBrejon

I kept digging with the new toolset I was mentioning and I was able to get almost where I wanted to be. Except the blues, which is still a stopper for me.

So that’s a no with respect to “modifying the look”, then.

Given that we have yet to find anyone actually formulating what is wrong in any rigorous manner, that implies that we cannot arrive at a solution. The problem is ill posed from the get go.

Knowing the little that I do about how complex the surface of forming a picture is, I’m going to say that authorship on open domain relative wattages prior to a pictorial depiction being algorithmically formed is nigh on impossible. Yes, we can make a case that some algorithms are more “less impossible”, but that fact remains; we are sitting here adding layers and layers and layers to libraries and architectures without even the remotest shred of a clue as to the veracity of what we are doing.

And the proof is in the pudding; look at the mess.

[daniele (Daniele Siragusano)]

@Troy_James_Sobotka

If we do not have a prevailing understanding of how excitation purity must attenuate in a pictorial depiction, we are saddled with an ill posed problem.

I think we do not need to know all the working in order to produce nice images and even very compelling heuristic. In the same way a cat does not know the laws of physics and still is able to navigate quite elegantly through the world.

But I agree our reasoning and understanding is by far not fundamental. And we need to seek for a better understanding.

For me RGB Tonemapping is a lucky accident which worked for a special case, but is by far not fundamental and also quite limiting in terms of achievable looks.
I am happy that ACES 2.0 has moved away from this plateau.

[Troy_James_Sobotka (troy_s)]

@daniele

I think we do not need to know all the working in order to produce nice images and even very compelling heuristic.

I don’t see anyone seeking much “working” at all, let alone “all the working”. Seeking one specific dimension that is causing absolute chaos in pictures, and has done so since the advent of the electrical sensor, as best as I can see, might be a good start.

@daniele

But I agree our reasoning and understanding is by far not fundamental. And we need to seek for a better understanding.

For me RGB Tonemapping is a lucky accident which worked for a special case, but is by far not fundamental and also quite limiting in terms of achievable looks.

If it is an accident, are we sure that we’ve gleaned all we can from the mechanisms? If this were the case, we could argue that the proof is not in the pudding.

If we are in agreement that as a design goal, it would be a good step if we can move away from the peculiar complexities of channel by channel distortions, then the next stage would be to design a potential solution with falsifiable claims. That’s the “Science” part one might argue.

Given that again, we are absolutely lost with respect to the excitation purity dimension, is it prudent to jump the shark and leap to another random model that is clearly and demonstrably inferior, to the point that conjectures can be made and falsified or verified? That would be like suggesting that there are problems with one architectural design of a building, and swapping out support structures with random guesswork. Where there is nothing falsifiable, there is no structure at all and only theist belief.

Getting back to the point, we are always enforcing some picture formation, either explicitly or implicitly. As such, and given how little we know, how little research is being done in this domain, and how poorly we understand things, does it make sense to try and argue for an iron fisted architecture and an arbitrary picture formation? Doubly so, one that many minds can falsify many stages of the premises of?

[daniele (Daniele Siragusano)]

@Troy_James_Sobotka

Getting back to the point, we are always enforcing some picture formation, either explicitly or implicitly. As such, and given how little we know, how little research is being done in this domain, and how poorly we understand things, does it make sense to try and argue for an iron fisted architecture and an arbitrary picture formation? Doubly so, one that many minds can falsify many stages of the premises of?

Absolutely! If we believe that we “have arrived” at a solution here, progress is doomed. I tried to make this clear with the ACES Meta framework.
I also agree that the current complexity is not justifiable, considering that the “heavy lifting” is still the job of the LMT.

[Troy_James_Sobotka (troy_s)]

@daniele

considering that the “heavy lifting” is still the job of the LMT.

The picture formation is plausibly doing the heaviest of lifting of formatting the pictorial spatial projection.

The “grading” is always in relation to this formation technique, and is only a small subject of the logic as best as I can tell?

@daniele

For me RGB Tonemapping is a lucky accident which worked for a special case

Curious why you’d suggest this, given that there are very clear trends being applied to the energy levels that provide insight? If it were purely an accident, it would seem fair to expect the sort of slop we get out of electronic renders and sensors, yet we do not see this manifest in a well designed system?

We are surrounded by accidental picture formation techniques that are all arguably rather catastrophic. It seems at least plausible that there’s more going on than a cursory review reveals?

[Derek]

@ChrisBrejon

I think a good way to think about this is using the Planckian locus/Kelvin temperatures to generate some fire simulations and making sure they do not come as “salmon pink” (I think Liam´s example above uses this already). It worries me that FX artists have to compensate the Picture Formation behavior in their scenes (e.g. the stimuli ain´t the image).

I think a good test image here could be the one of real fire by @Thomas_Mansencal. Here we can see it looking salmon-pink in ACES 2.0

compared to OpenDRT Default Look where it is fire-colored fire.

Regarding blues, I found this comparison interesting. Note in particular the blues on the ceiling which look purple in ACES 2.0:

compared to OpenDRT Default Look:

I’ve tried to do some analysis of this in CG, and from what I could see, I’d say that the issue with ACES2.0 is not so much that red goes to pink at high exposure. Of course it does. Rather the issue is how red has such a dominant overpowering effect on other colors. You might say there is too much weight given to red. So red mixing with yellow does not produce orange, but rather the red overpowers it making it salmon. Similarly in the blues, the red in the purple overpowers the blue. That’s all very unscientific hand-wavy observations, that I’m sure I’m not describing in the best way, but hopefully it makes some sense.

[TooDee (Daniel Brylka)]

Hi Chris,

I am playing around with Blender & ACES 2.0 as the Blender 5.0 Alpha has direct support for ACES 2.0, next to Standard, Filmic and AgX. I found this out over this video:

Blender 5.0 has ACES support, kind of...

Before I understand how ACES 2.0 will be implemented in Blender 5.0,
I was setting up a Blender scene in the release version 4.5.2 but with the official ACES 2.0 OCIOv2 config.

I basically re-used the scene from this article (Blender 4.5 & ACES 2.0 – Brylka – TooDee – PanicPost), but switched all the lights off and changed all the shader to emission shaders with the same values.

I rendered two images:

1. working colorspace Lin-Rec.709, shader values at 0.900, view transform standard
2. working colorspace ACEscg, shader values again at 0.900 (so intentionally a lot more intense “colors”), view transform ACES 2.0 SDR 100nit (Rec.709) - to make the images similar “bright” I raised the overall scene exposure by 1 1/3-stop.

So far so good or not good…
While I tried to find a similar exposure for the ACES 2.0 render to match more or less the first render, I noticed something in the reflection of the blue sphere.

And I wanted to ask you @ChrisBrejon if you can confirm the problem with your DCC app as well.
You made so many exposure sweeps, I guess you would have seen it too at some point.

It could be a glitch in Blender or my MacOS setup, but I see the same happening in Nuke with ACES 2.0 (rev.060 from the LUT bake version)

The reflection of the magenta sphere reflects as dark blue and at then it “flips” to a light blue/magenta.

I can share the .blend file if someone wants to try it too.

PS:
I remembered now a discussion in the ODT-VWG threads… and a plot… is the problem maybe related?

ACES 2.0 CAM DRT Development

Is the objective now to match the Reveal rendering?

(the plot of the blue/yellow star image)

Update;

I tried again the same scene on a MBP with XDR display and here the blue does not flip as on the Mac Studio with Apple Studio Display. So there might be another problem with MacOS, Blender happening that I was not aware of.

[chuckyboilo (Charles Boileau)]

Great read. I never hoped to open such a “can of worms”.

Thanks!

[Troy_James_Sobotka (troy_s)]

@Derek

I think a good test image here could be the one of real fire

It’s a picture.

It is no more a “real fire” than the EXR is a “test image”; until the relative colourimetric wattages are formed from the open domain relative wattage values into a proportionate picture signal, there’s no “image”. To ascribe the “image”, or more specifically a pictorial depiction, to some jumble of colorimetry in an EXR, is a grave misplacing of “where the picture is”.

Once we have a planar presentation of a pictorial depiction formed in stimuli, we can begin to evaluate how the heck we think we “see” what we “see”. So how do we “see” the “fire”? Here’s a formed picture from a newscast, for example. Given there wasn’t a civil uprising over the newscast, we can infer the audience parsed the articulation as “fire”.

So how the heck do we cognize “fire” in a two dimensional surface array of relative wattages within a formed pictorial depiction?

@Derek

Note in particular the blues on the ceiling which look purple in ACES 2.0:

It might be of sufficient utility to simply ask ourselves what we think we see in these two pictures, as they are exemplary of a few ideas:

1. What do we think we “see” across the two pictures?
2. Do we cognize forms / shapes oriented in pictorial spaces across the two pictures?
3. What are the “base colours” of the forms between the two pictorial depictions?
4. What are the “surface qualities” of the forms across the two pictures?
5. Are the “base colours” of the cognized forms “different” in some way between the two pictorial depictions?
6. Are the “surface qualities” cognized from the unique algorithms of the pictorial formations different between the two pictorial depictions?
7. If one does cognize a difference in the “surface qualities” of the formed pictorial depictions, does this imply a different pictorial spatial projection being shaped by the differing algorithms?
8. What would be a cognized “best guess” of the “reflected form colour” depicted in the “surface quality”? IE: Not the “base colour” of the form depicted, but the “base colour” of whatever is present in the “reflection” off of any “surface quality”?

[Derek]

I believe I follow. Essentially working in a wider color space magnifiers existing problems without providing creative benefit. Is that right?

I asked this question in a discussion post. Here’s some replies. Curious how you would respond

A downside could be …imagine you get sent a plate / HDRI in ACEScg and you want to use it in linear-Rec.709. After a colorspace transform from ACEScg to linear-Rec.709 you could end up with negative values… as an example.

The problem with 709 as working space potentially is having negative values at that particular moment in the chain, because that’s where you do your operations, regardless of what transform happens before or after. And multiplying a negative value with a number larger than 1 pushes it even further out of the gamut. So for grading or compositing with some tools/operations this can be problematic.

This would apply in a VFX or grading workflow where you could be having files from an outside source in these wider gamuts . . However, in a CG animation pipeline you can, of course have everything in the same color space. One advantage I see there of having the working space be Rec 709 is that many renders calculate things like Kelvin temperature and melanin in Rec 709, producing incorrect colors when the working space is ACEScg

[Thomas_Mansencal]

@Derek

One advantage I see there of having the working space be Rec 709 is that many renders calculate things like Kelvin temperature and melanin in Rec 709

You can’t really make accurate fire simulation with sRGB, the primaries cannot encode the lower Kelvin temperatures. This was one motivation to adopt a larger working space in studios I worked for.

[ChrisBrejon (Christophe Brejon)]

@TooDee

And I wanted to ask you @ChrisBrejon if you can confirm the problem with your DCC app as well.
You made so many exposure sweeps, I guess you would have seen it too at some point.

Hey, were you able to double-check your setup out-of-curiosity ?

I did try in Guerilla render and I can reproduce the polarity flip you were mentioning (I think).

I used ACEScg magenta (1,0,1) and blue (0,0,1) “primaries” for the shading of the spheres and ACEScg as a rendering space. The plane ground is a 0.18 midgrey.

I would be curious if anyone has seen a similar behavior and/or a potential fix (LMT ?) for it.

Regards,

Chris