Some colours still appear too bright in SDR (for example yellow and magenta in ACES OT VWG Sample Frame - 0029). I had “Encode as BT.2100 PQ” checked to compare SDR and HDR on the same monitor.
My theory is the following:
The compression of lightness J is much stronger in SDR than in HDR. Maybe some colours that tend to be bright (yellow, orange, magenta, cyan) need some additional compression of lightness J.
There is also the possibility that my monitor is just deceiving me.
Looking again at the chromaticity plot from @TooDee’s star image , it seems to me that the sharpness of the “double back” in the plot is a result of two things.
Firstly, and probably most importantly, it is not actually as sharp as it looks, because we are viewing a 2D projection of a line which in 3D space actually curves down and back in.
Secondly, the corner is sharpened by the fact that in the blue corner, the blue primary of the limiting Rec.709 gamut and the reach AP1 gamut are very close to each other. The compression curve uses threshold and limit parameters calculated as:
limit = \frac{M_{reach}}{M_{boundary}}
threshold = \frac{1}{limit}
This means that when the reach and boundary M values are close, as they are in the blue corner, then limit is only just above 1.0 and threshold is only just below it, resulting in quite a sharp transition into compression.
I remember @priikone added this varying threshold to address a particular issue, but I cannot remember what that issue was. In any case, because other things have changed since then, it may be worth revisiting it to see if the issue it solved still exists, and whether what its negative impact in the blue corner is a good tradeoff against its benefits elsewhere.
The issue was that if the limt is close to 1.0 and the threshold is constant (like 0.75), it wouldn’t compress much at all. It was causing artifacting in HDR as it was just clipping rather than compressing. So even a small amount of compression fixed the artifacts (I think we tried to look at some of those artifacts in one of the meetings also).
Presumably the artefacts are caused by the imprecision of the gamut approximation, followed by final clipping. Because in theory, if things were precise “not compressing much” would be the right thing to do, as only a very small amount of compression would be needed to compress the AP1 boundary to the limiting gamut boundary, because they are already close.
Or maybe moving the threshold is about affecting the slope of the compression curve as it hits 1.0.
This version fixes the white point used in the computation of various tables, especially the reach table and the AP1 gamut cusp table. The white point for them is now the ACES white rather than D65.
As a result the inverse improved, so this version now reduces the cusp smoothing value and smoothing factors without negatively impacting the inverse. The end result is slightly less clipping in the forward direction. There’s not really a visible difference to v056.
@TooDee’s stars chromaticities v057 vs v056 in sRGB output:
Following @priikone’s suggestion in last night’s meeting, I’ve added optional Björn Compress Mode to the Nuke and DCTL implementations of my LMT.
They currently still use the LMS matrix from v55, but since with Björn compression they will be using a JMh space which doesn’t match the one in the DRT anyway, the exact LMS matrix didn’t seem important.
The compress mode certainly seems to reduce the darkening of the inner area of the blue star.
I also noticed that we could also reduce the lower hull gamma from 1.14 to 1.139 which reduces clipping even more, without impacting the inverse. So maybe in next version.
After yesterday’s meeting I went over hellwig model again , and I was able to simplify the color model itself it to just a few nodes, I am getting the same values as v55 in DCTL
I wanted to do this to understand the behaviors. If you see the nodes the color model can be explain the following way:
convert to XYZ
convert to the LMS space which is technically just an RGB space with a red primary slightly outside XYZ, a green primary so far outside that it breaks the xyY calculation, and a blue primary slightly outside XYZ.
re-normalize achromatic using gains
RGB values are transformed using a sigmoid function.
The original model likely only dealt with values between 0 and 1 so not a huge problem for that case. We can achieve a similar effect with a simple power function and plus some gain, although this gain gets canceled later on in step 7.
This step, like any adjustments made to individual color channels,this is what cause hue shifts. It breakes values outside the RGB range, and while DRT helps a bit, it doesn’t fully address the issue.
Similar to other per-channel adjustments, this step pushes colors towards the primaries of the LMS/RGB color space its applied in. When compared to common display standards (P3, Rec.709, Rec.2020), the color space used here has more magenta in the blue and red channels, and more cyan in the green channel. This significantly shifts colors towards magenta and cyan which is the opposite of aesthetically pleasing results, it not as noticeable due to the rest of the DRT not been per-channel.
Another drawback of this step, like with other per-channel adjustments, is a slight tendency to push colors towards achromatic.
Rotate the cube to be up-right, plus scale the 3 channels to be better spread around.
convert to a cylindrical coordinates space.
scale achromatic to make J channel 1 = Luminace 1, for achromatic, the rest still scaled differently.
That is an excellent analysis @jpzambrano9 Well done ! I am pretty sure many people on this forum will appreciate this deconstruction of the model.
I could not help but think that per-channel was behind this behavior when I heard Pekka´s question in yesterday´s meeting (around 30:00).
Why is it that with this transform the skew is always towards the secondaries ?
If I recall correctly, the first person who explained to me this behavior was Troy and even if I did not understand all the maths behind it, I put it in my article since it seemed relevant.
That’s really nice and I can confirm that it produces correct values (after a little bit of tweaking). Only difference I notice is with extreme input values that in the DRT would need the AP1 clamp. Those don’t come out the same. In the DRT (without the AP1 clamp) the M collapses to 0 but in your node the M is a positive value (which I consider better behavior). The J however collapses to 0 in both of them.
This should be Hellwig\ RGB_{aw/wc} rather, the sigmoid compression is applied to LMS cone space values, not to the JMh correlates. Also worth noting that the model was fitted whilst accounting for the clumping. The model is designed to predict the appearance of patch of colours under different viewing conditions, it was not built for colour rendering and it should be expected that changing things inside for colour rendering might introduce/uncover undesirable behaviours.
Yes, the mistake @priikone picked up only affected hellwig_lib_v057.h. The other v57 DCTL files are unchanged.
Thanks for that. I don’t have an OpenCL system here to test. When I had one before I always found it was far more picky about DCTL syntax than the others.
I pushed CAM DRT v058 with some new parameter values to my repo. I’ll create a pull request for Alex later after we’ve discussed some things in the meeting.
One thing we’ve talked privately was P3D65 inverse. Previous versions haven’t inverted P3 as well as Rec.709 and to get it inverse better we’d have to increase cusp smoothing, rather than what I’ve been doing lately, which is to reduce it to reduce clipping. This version should bring P3 inverse to same level as Rec.709, but it does come with a bit more clipping in the forward direction in Rec.709.
Param changes (no code changes):
Change power of powerP compression from 1.2 to 1.0
Change Lower hull gamma constant from 1.14 to 1.145
Change Smooth cusps value from 0.18 to 0.12
Change Smooth J value from 0.02 to 0.0 (becomes possible to eliminate)
During last night’s meeting, @jpzambrano9 pointed out that the reach cusp (AP1) table in my DCTL did not quite match the table in the Blink. I realised that my Python which generates the tables was still incorrectly using D65 white for the AP1 reach, despite that being fixed in the Blink a few versions ago.
I have just pushed a commit to fix that in the v58 DCTL.
