ACES 2.0 CAM DRT Development

[daniele (Daniele Siragusano)]

@daniele

The mastering colour space can easily accommodate different white points by scaling its RGB primaries differently.

I should have been more clear here. I mean, you literally scale the primaries of your display.
So if you define a P3D55 mastering projector, you calibrate the projector’s primaries so that 1,1,1 lands at D55 48 nits. That is conceptually the definition of mastering colour space.

Of course, you could be lazy and don’t bother to change your projector’s calibration from P3D65 to P3D55, but then the projector you are using is not the mastering projector but the encoding projector. And in fact, the white would be needed to be scaled down to fit inside the encoding gamut. (so you better scale up the lamp/laser power ).
Another issue with the lazy way would be that your output RGB scopes would always have that offset built in. So if you desaturated an image, the RGB parade would still show an offset. And that is not so nice for the colourist , because you’d expect that your mastering white point produces equal r=g=b values on the scopes.
I always advise calibrating the mastering display to the intended mastering colour space (including white of course) , all professional monitors and projectors can store presets for different colour spaces with different white points.

[nick (Nick Shaw)]

While working on the DCTL implementation, it occurred to me that the conversion from mono J to linear for the tone curve must be a simple function, that we are applying via a complex method (understandably for development purposes).

I wrote a Python implementation of the conversion, and then tried to fit a curve to it. I hoped it would be a simple power curve, but although close, it isn’t. I then tried an offset power curve of the form:

y = \frac{(x+o)^p - o^p}{(1+o)^p - o^p}

It’s not quite right, but I found a pretty close fit:

Is that (or something similar) close enough for our purposes? Perhaps @luke.hellwig can tell us what the actual curve should be.

[priikone (Pekka Riikonen)]

I managed to get a new simpler version of the chroma compression working. It’s available in my repo for testing as a prototype version CAM DRT v032-pex.

New in CAM DRT v032-pex prototype

• New chroma compression algorithm, old one is removed
• Cusp smoothing is now enabled by default
• Small change to custom LMS primaries
• Small changes to LMS compress mode to avoid NaNs/Infs
• Gamut mapper parameters changed
• ZCAM is removed
• Linear extension is removed

The New Chroma Compression (in-gamut compression)

The new algorithm is simpler than the old one and has only one curve as opposed to three curves the previous algorithm had. The steps of the new algorithm are as follows:

• Scale down M (colorfulness) by tonescaledJ / origJ
• Normalize M to compression gamut cusp (becomes hue-dependent)
• Compress M with a compression curve. Compression is increased as tonescaledJ increases to create the path-to-white.
• Denormalize M with the gamut cusp
• Apply global saturation to taste, boosting saturation in shadows more

The M is normalized to the cusp of a compression gamut. The compression algorithm then compresses the colorfulnes in 0-limit range. It does not compress anything beyond the limit. This makes this curve more controllable than the previous one and it protects pure colors well. Forward direction has better yellows than previously which helps with rendering of fire/pyro, etc. Path-to-white is IMO better than in the previous algorithm. Path-to-black is on par with the previous one with slightly more colorful noise.

The Compression Gamut

The default chroma compression gamut is always Rec.709 but I think it could also be something else, including a custom gamut specifically chosen for the compression use case. This gamut doesn’t change even when limiting/encoding primaries are changed. This means it keeps the range that it compresses (0-to-limit) always the same regardless of the display gamut. This may or may not be a good idea…

The gamut cusp is always scaled with an eccentricity factor (ET). The current implementation has 3 ETs to choose from: CAM16, Hellwig2022 and Custom. The default is Custom and provides the best forward and inverse directions of the three. Without applying the eccentricity factor it seems the shape of the resulting compression fits the display gamut poorly.

The 3 compression parameters are limit, compression strength, and compression expansion rate.

The Rec.709 Inverse

Following images show the inverse compared to CAM DRT v031 (second image):

Following images show the inverse with CAM16 and Hellwig2022 eccentricity factors applied:

Following image show what the inverse looks like when eccentricity factor is not applied in compression (all else being equal):

SDR/HDR Rendering

The match between SDR and HDR is decent but not perfect. Shadows can be more colorful in HDR compared to SDR. Needs more testing and adjustment. Path-to-white for normal range of colors appears to be almost identical in SDR and HDR, which is an improvement over the previous chroma compression, IMO.

Purer colors can now appear even more pure in HDR than before because chroma compression no longer compresses as far out as it did before. Pure red in particular seems to come out really hot on HDR RGB screen (with either P3 or Rec.2020 limited primaries).

Example Images

First image is CAM DRT v032-pex, second image is CAM DRT v031 for comparison:

[Derek]

Oooh! That’s looking really nice!

[priikone (Pekka Riikonen)]

I opened a pull request for CAM DRT v032 for @alexfry, also available in my repo.

This version is otherwise identical to the prototype version above but the SDR/HDR match has been improved a little bit, and colorfulness of noise has been reduced.

Here’s a recap what has changed in v032:

• New chroma compression algorithm, old one is removed
• Cusp smoothing is now enabled by default
• Small change to custom LMS primaries
• Small changes to LMS compress mode to avoid NaNs/Infs
• Gamut mapper parameters changed
• ZCAM is removed
• Linear extension is removed

[alexfry]

The PR has been merged, and the LUT Candidates repo has been updated with bakes for v032.

GitHub - ampas/ACES_ODT_Candidates

So far the SDR to HDR match is looking pretty sane to me.

[nick (Nick Shaw)]

I am going to wait until people have tested the LUT bake versions for a bit before porting this new stuff to DCTL. If we are confident that this new direction is the one we are taking, I will work on it then.

[Shebbe]

I have the new MacBook now and able to view SDR and HDR. A very crude run through the exr set tells me that they match very well on most images especially the color/hue. The contrast/dynamic range is a bit weird to experience viewing both at the same time in same conditions on a 16" display. I have to get used to the presentation of HDR on reference monitor in general, but there are definitely some things that stood out on which I want to write about in more detail when I have more time.

[jeffrey.d.mathias]

Have given a quick look to all 6 variations, but have not yet looked at the inverses.

Not to contradict Shebbe, but I have found some differences between
ACES2 Candidate CAMDRT rev032 Rec709,
ACES2 Candidate CAMDRT rev032 Rec2100 (Rec709 sim), and
ACES2 Candidate CAMDRT rev032 Rec2100 (P3D65 1000nit Limited)

In particular the CIE Chromaticity scope shows some variations of which cause unknown, but differences worth pointing out.
The Vectorscope and color charts seem to show some differences in saturation for these three which were not the same with comparisons of earlier versions.
Also, it is very difficult to evaluated 3-D with a 2-D scope.
Also I am not sure if being in the 709 or P3 color space is behaving differently or as it should.

Otherwise the images (all sample frames) look good and maybe consistent between SDR and HDR, but this has only been a very quick look. More time is needed to compare and I would feel better without the issues mentioned above.

I understand Nick not wanting to rush ahead with DCTLs and would encourage closer inspection of what the compression and model is doing in HDR and if it is behaving as expected and not messing with saturation or hue.

[nick (Nick Shaw)]

Could somebody sanity check me here! I am trying to work out why my DCTL isn’t matching the LUT bake, and I 've found a few things.

1. A few of the parameters seem to have changed slightly between v30 and v31 (at least in the LUT bake project) so it does not seem to be the case that v31 with the 7-axis values set to detents is identical to v30. I have changed the parameters in my DCTL accordingly and get a better match. But I’m wondering now if the LUT bake is using out of date parameter values.
2. My DCTL is using my quadratic solve based gamut compression. But looking at the Blink in the v31 LUT bake Nuke project, it does not appear to be using that. The blink in Alex’s repo does, so there seems to be a mismatch.
3. The group nodes in the rev31 LUT bake Nuke Project do not seem to include the sixAxisCompression, which suggests they may not actually be rev31.
4. All the Blink nodes still seem to be using [4200, -1050] for the white point in the LMS matrix calculation. Should those not have been changed to Equal Energy White? Or maybe that decision came after rev31.

I won’t push my updated DCTL yet, as I have a feeling I may be tracking the wrong version of the Blink.

EDIT: It appears I may have been working from an earlier commit of the rev31 LUT bake project. The one in the repo currently matches the parameters I originally used, and also includes sixAxisCompression.

[priikone (Pekka Riikonen)]

I’m not sure about how the v31 LUTs were baked, but the CAM_DRT_v031.blink is the correct blink code for v031. It is otherwise identical to CAM_DRT_v030.blink but it has the hue-angle gamut mapping compression parameters. However, those parameters in v031 node should not be in use, so in practice v030 and v031 are identical (rendering should be identical).

v031 node still uses the wrong illuminant for the LMS matrix. This was fixed in v032.

[nick (Nick Shaw)]

I think the rev31 LUT bake Nuke project may not have been properly updated in the repo until the rev32 commit yesterday.

[alexfry]

Yeah, that was a bit of a weird sequence of events thing.

I baked the v031 luts, but hadn’t hit save in Nuke before I hit commit on the repo. The .autosave was in the correct state, but not the main .nk.

I corrected that before yesterday’s commit, which is why the v031.nk updated yesterday. The relationship between the v031 .nk and the v031 LUTS should be correct.

[Derek]

I don’t know if it’s at all meaningful, but looking at the image set on my M1 Macbook Pro with the OCIO baked LUTs for rev032, I’m seeing a pretty decent match between SDR (P3D65 Limited) and HDR (P3D65 1000nit Limited). However when I instead compare P3D65 Limited to P3D65 540nit Limited, the 540nit HDR looks a lot more saturated. A good example to look at is ACES OT VWG Sample Frame - 0007.

FWIW, I don’t see this in rev031.

[Shebbe]

I just looked at HDR P3D65 540nit and HDR P3D65 1000nit and I experience the same thing. The reds are more saturated in 540nit than in 1000nit. Blues are closer, greens couldn’t see a noticeable difference. They match a lot more in rev031. Overall rev031’s 540 nit more closely matches rev032’s 1000nit, so not sure if the rev032 P3D65 540nit is as it’s intended to look.

[jeffrey.d.mathias]

Confirm what both Derek and Shebbe are seeing in that the 540 nit shows more saturation than the 1000 nit, 709sim and SDR. Is this related to the max luminance somehow?

Also red Christmas now looks to be orange Christmas for every case.
And I and seeing Red neon in HDR look more pink in SDR and sim.
Also noted that the powder blue shirt in Isabella (good in SDR and sim) gets some magenta tint in HDR. Note that for the sim and HDR I am using split screen on the same monitor.
More to come as studied further.

[priikone (Pekka Riikonen)]

Last night I compared rec.709 sim and rec.2100 again after a week or so, and I was immediately seeing the difference. HDR clearly has higher overall saturation level. The match is worse than in v031 which I considered quite a good match.

This needs more investigation, testing and tweaking to learn more, but I am finding it much more difficult the create the match. This reminds me of how it was also difficult with version v026 that had the original curve, similar to what the original highlight desat curve was.

Following image is showing the v026 chroma compression curve for 100 and 1000 nits (in all graphs the higher curve is the 1000 nit curve):

The match was improved to v028 with a new curve. Following image is showing the v028 chroma compression curve for 100 and 1000 nits:

The curve in v028, and in subsequent versions until v032, was the key creating better SDR/HDR match. I found that it was important to keep the curves at and around the middle gray (the red vertical line) at similar rate of change. Darker colors wouldn’t match at all with v026 curve but matched immediately with v028 curve. The v026 curve, while overall was more colorful, was less colorful in shadows in HDR (as can be seen in the v026 curve the HDR curve has faster rate of change in shadows). In v028 curve they’re closer in shadows. So only thing I had to adjust was global saturation to get the match to overall similar level.

In v032 all this went away and the algorithm now starts by dividing the tonescaled lightness with the original lightness. The following image shows how colorfulness will change with this, again for 100 and 1000 nits lightness (image shows tonescaled J divided by original J):

These curves tells us that the HDR image will be more colorful, including in the shadows. This happens because the middle grey is lifted in the tonescale. I quickly tested last night also lower lift in the middle grey, and lowering the lift from ~15 nits (from 100 nits to 1000 nits) to ~12-13 nits created much better overall match.

(I’m also wondering (and not knowing) whether HK effect plays a role here too since not only is the exposure lifted for HDR it’s also more colorful so it appears even brighter? My eyes seem to agree.)

Now, these curves are only one part of the final chroma compression which includes another curve for the inner-gamut compresion, but the fact that they differ so much at and around middle grey I think is making it harder to create the match simply by adjusting saturation.

I don’t have, at this very moment, a solution to bring the match closer. I will hope to continue testing and tweaking and see if I can get them closer without adding too much complexity.

[jeffrey.d.mathias]

===========================================================
Purpose - to compare HDR and SDR for version 032 ACES2 ODT

Systems A and B described at end.
As an initial check, both systems were compared using SDR for the Rec709 v032 and were found to be closely identical for all images. Then both were compared using HDR for the rec2100(rec709sim) v032 and were found to be closely identical for all images. And the same done with rec2100(P3D65 540nit) with similar results.

For the following comparison duplicate frames were made with rec2100(rec709sim), rec2100(P3D65 540nit), and rec2100(P3D65 1000nit) and viewed simultaneously using split-screen with Computer A. Computer B was set to rec709 and viewed those images that way. So each frame had four views displayed simultaneously (3 in HDR and 1 in SDR) on two screens and evaluated.

Also as a check, rec2100(rec709sim) and rec2100(P3D65 540nit) were run on Computer B in HDR and rec709 on Computer A in SDR. This gave identical results to that above.

=============
Observations:

76 images were compared from the ACES2 frame samples.
Overall SDR and sim matched well unless noted otherwise.

In general HDR had more contrast as expected, but also showed more saturation with the 540nit group tending to have the most saturation, too much at times (as in 0064.)

Neon reds looked to be weak and pink in SDR.

In particular frame 0012 note the red letters of the Hollywood Museum sign (is this even the correct red?)… and red Christmas goes orange (0060.)

Found some clipping in 0017 in SDR.

Some had issues with blues see 0033, 0035, 0043 and 0067 below.

There looked to be more green in HDR of 0041.

Above are some summaries of the notes listed for each frame below, a blank note means SDR and HDR looked close or as expected.

001
002
003 neon red in HDR is neon pink in SDR and sim
004 SDR and sim match and HDR looks a bit more red more sat
005 SDR and sim match and skin looks better than HDR which might be more red
006
007 SDR and sim match with SDR a bit nicer and HDR too red/orange
008 here SDR and sim match and look less saturated than HDR
009
0010 HDR orange a bit more saturated
0011
0012 HDR has better red in signs which looks a bit orange and lower sat in SDR and sim
0013 reds all close
0014
0015 the sim cyan looked a bit weaker otherwise SRD was a bit greener cyan than HDR
0016 SDR and sim match but red neon is more pink and green neon a bit weaker than HDR
0017 SDR and sim weaker or less sat than HDR plus SDR has grey patch in sun (clipping?)
0018 HDR looks to have more sat
0019
0020
0021 HDR has a bit more sat
0022
0023
0024
0025 Red neon in HDR is pink in SDR and sim
0026 HDR has more sat
0027
0028
0029
0030 bananas and melon look weak in SDR and sim and more sat in HDR
0031 rather than SDR and sim looking weak HDR might be more sat
0032 rather than SDR and sim looking weak HDR might be more sat & Red neon different
0033 SDR and sim look grey (Sim more so) might be sat difference in blues
0034
0035 SDR and HDR match better than sim which looks grey or has magenta in blues
0036
0037
0038 neon seems to match better in this frame
0039
0040
0041 more green in HDR
0042 HDR has more sat
0043 blue in upper left sky of sim not as blue as others
0044 SDR and sim match HDR has more sat and neon name on building different (more orange than pink)
0045
0046 blues differ in all three
0047 SDR and sim look weak next to HDR (just not a lower dynamic scene)
0048 SDR and sim look weak next to HDR (just not a lower dynamic scene)
0049
0050
0051 is more contrast or luminance of HDR adding sat?
0052 is more contrast or luminance of HDR adding sat?
0053
0054
0055
0056 Red neon in signs of HDR goes pink in SDR and sim
0057 amazing close match, however screen seems shifted to purple
0058 all match with the same sat/contrast differences in skin
0059 all look orange and maybe not enough red for what has been called red Christmas, but they match
0060
0061
0062
0063 the red, blue green only work in HDR, note that the chart looks a bit weak in SDR and sim
0064 HDR looks too much sat, enough to not look real
0065 SDR and sim match but tail lights differ from HDR
0066
0067 I think the shirt is supposed to be powder blue which looks a bit magenta in HDR
0068
0069 immediately noticeable, the ferris wheel lights differ between HDR and SDR, sim
0070
0071 again the red neon
0072
0073
0074
0075
0076 again HDR is too much

==============================
Purpose - to check inverse IDT

A first node with ACES transform ODT v032
A second single node with ACES transform Input as inverse032 and Output as v032
checked for Rec.709, Rec.2100(709sim), and Rec2100(P3D65 1000nit)

Note that if the order of the nodes is reversed all fail without question.

Instead of listing a bunch of notes (available if desired) will just say that the inverse does not look to work well.

==========
Equipment:

Computer A - for HDR and SDR
AMD Threadripper 1950X on Asrock Fatal1ty X399
Windows 10 Pro 64-bit 21H2
AMD Radeon Pro WX 8200
BMD Decklink 4K Extreme 12G
LG C8PUA OLED TV calibrated & checked with Xrite i1Display Pro
Davinci Resolve 18.1.4 build 9

Computer B - for HDR and SDR
Apple Ipad Pro M2 12.9"
DaVince Resolve for iPad (latest)
Note: on the iPad for some reason files cannot be placed into the directory for ACES transforms so DCTL were used instead of ACES Transforms by commenting out the first line of code.

[priikone (Pekka Riikonen)]

I opened a new pull request for @alexfry for CAM DRT v033, also available in my repo.

I spent some time looking and improving the SDR/HDR match. This version should have a match that is on par with v31, the last version that had a good match. By scaling the compression curve (aka. tonescaled J / original J) in non-linear fashion I was able to improve the match.

This version adds also a new custom chroma compression gamut. It can be adjusted inside the kernel parameters as xy coordinates. Previous version was using rec.709 for the compression but as agreed it is better to define our own.

EDIT: I closed the pull request and opened a new one (#16). I noticed I had left wrong gamut mapper parameters in the node. No other changes.

[nick (Nick Shaw)]

@jeffrey.d.mathias

Purpose - to check inverse IDT

A first node with ACES transform ODT v032
A second single node with ACES transform Input as inverse032 and Output as v032
checked for Rec.709, Rec.2100(709sim), and Rec2100(P3D65 1000nit)

Note that if the order of the nodes is reversed all fail without question.

Instead of listing a bunch of notes (available if desired) will just say that the inverse does not look to work well.

I think you may be expecting too much of the inverse. It does not aim (nor would it really be possible) to take ACES2065-1 values transformed through a forward DRT and revert them back to the original ACES2065-1 values.

Rather the intention and purpose of the inverse is to be able to take display-referred image data, either from existing display referred imagery, or from processing scene data through a different rendering, and invert that to a limited set of ACES2065-1 data such that when subsequently processed through the forward transform it lands back at the previous display-referred values.

TLDR: inverse followed by forward DRT should be (close to) a NoOp for input in the 0-1 range. Forward DRT followed by inverse is not expected to produce a NoOp for all input values.