I think the rev31 LUT bake Nuke project may not have been properly updated in the repo until the rev32 commit yesterday.
Yeah, that was a bit of a weird sequence of events thing.
I baked the v031 luts, but hadn’t hit save in Nuke before I hit commit on the repo. The .autosave was in the correct state, but not the main .nk.
I corrected that before yesterday’s commit, which is why the v031.nk updated yesterday. The relationship between the v031 .nk and the v031 LUTS should be correct.
1 Like
I don’t know if it’s at all meaningful, but looking at the image set on my M1 Macbook Pro with the OCIO baked LUTs for rev032, I’m seeing a pretty decent match between SDR (P3D65 Limited) and HDR (P3D65 1000nit Limited). However when I instead compare P3D65 Limited to P3D65 540nit Limited, the 540nit HDR looks a lot more saturated. A good example to look at is ACES OT VWG Sample Frame - 0007.
FWIW, I don’t see this in rev031.
1 Like
I just looked at HDR P3D65 540nit and HDR P3D65 1000nit and I experience the same thing. The reds are more saturated in 540nit than in 1000nit. Blues are closer, greens couldn’t see a noticeable difference. They match a lot more in rev031. Overall rev031’s 540 nit more closely matches rev032’s 1000nit, so not sure if the rev032 P3D65 540nit is as it’s intended to look.
Confirm what both Derek and Shebbe are seeing in that the 540 nit shows more saturation than the 1000 nit, 709sim and SDR. Is this related to the max luminance somehow?
Also red Christmas now looks to be orange Christmas for every case.
And I and seeing Red neon in HDR look more pink in SDR and sim.
Also noted that the powder blue shirt in Isabella (good in SDR and sim) gets some magenta tint in HDR. Note that for the sim and HDR I am using split screen on the same monitor.
More to come as studied further.
Last night I compared rec.709 sim and rec.2100 again after a week or so, and I was immediately seeing the difference. HDR clearly has higher overall saturation level. The match is worse than in v031 which I considered quite a good match.
This needs more investigation, testing and tweaking to learn more, but I am finding it much more difficult the create the match. This reminds me of how it was also difficult with version v026 that had the original curve, similar to what the original highlight desat curve was.
Following image is showing the v026 chroma compression curve for 100 and 1000 nits (in all graphs the higher curve is the 1000 nit curve):
The match was improved to v028 with a new curve. Following image is showing the v028 chroma compression curve for 100 and 1000 nits:
The curve in v028, and in subsequent versions until v032, was the key creating better SDR/HDR match. I found that it was important to keep the curves at and around the middle gray (the red vertical line) at similar rate of change. Darker colors wouldn’t match at all with v026 curve but matched immediately with v028 curve. The v026 curve, while overall was more colorful, was less colorful in shadows in HDR (as can be seen in the v026 curve the HDR curve has faster rate of change in shadows). In v028 curve they’re closer in shadows. So only thing I had to adjust was global saturation to get the match to overall similar level.
In v032 all this went away and the algorithm now starts by dividing the tonescaled lightness with the original lightness. The following image shows how colorfulness will change with this, again for 100 and 1000 nits lightness (image shows tonescaled J divided by original J):
These curves tells us that the HDR image will be more colorful, including in the shadows. This happens because the middle grey is lifted in the tonescale. I quickly tested last night also lower lift in the middle grey, and lowering the lift from ~15 nits (from 100 nits to 1000 nits) to ~12-13 nits created much better overall match.
(I’m also wondering (and not knowing) whether HK effect plays a role here too since not only is the exposure lifted for HDR it’s also more colorful so it appears even brighter? My eyes seem to agree.)
Now, these curves are only one part of the final chroma compression which includes another curve for the inner-gamut compresion, but the fact that they differ so much at and around middle grey I think is making it harder to create the match simply by adjusting saturation.
I don’t have, at this very moment, a solution to bring the match closer. I will hope to continue testing and tweaking and see if I can get them closer without adding too much complexity.
===========================================================
Purpose - to compare HDR and SDR for version 032 ACES2 ODT
Systems A and B described at end.
As an initial check, both systems were compared using SDR for the Rec709 v032 and were found to be closely identical for all images. Then both were compared using HDR for the rec2100(rec709sim) v032 and were found to be closely identical for all images. And the same done with rec2100(P3D65 540nit) with similar results.
For the following comparison duplicate frames were made with rec2100(rec709sim), rec2100(P3D65 540nit), and rec2100(P3D65 1000nit) and viewed simultaneously using split-screen with Computer A. Computer B was set to rec709 and viewed those images that way. So each frame had four views displayed simultaneously (3 in HDR and 1 in SDR) on two screens and evaluated.
Also as a check, rec2100(rec709sim) and rec2100(P3D65 540nit) were run on Computer B in HDR and rec709 on Computer A in SDR. This gave identical results to that above.
=============
Observations:
76 images were compared from the ACES2 frame samples.
Overall SDR and sim matched well unless noted otherwise.
In general HDR had more contrast as expected, but also showed more saturation with the 540nit group tending to have the most saturation, too much at times (as in 0064.)
Neon reds looked to be weak and pink in SDR.
In particular frame 0012 note the red letters of the Hollywood Museum sign (is this even the correct red?)… and red Christmas goes orange (0060.)
Found some clipping in 0017 in SDR.
Some had issues with blues see 0033, 0035, 0043 and 0067 below.
There looked to be more green in HDR of 0041.
Above are some summaries of the notes listed for each frame below, a blank note means SDR and HDR looked close or as expected.
001
002
003 neon red in HDR is neon pink in SDR and sim
004 SDR and sim match and HDR looks a bit more red more sat
005 SDR and sim match and skin looks better than HDR which might be more red
006
007 SDR and sim match with SDR a bit nicer and HDR too red/orange
008 here SDR and sim match and look less saturated than HDR
009
0010 HDR orange a bit more saturated
0011
0012 HDR has better red in signs which looks a bit orange and lower sat in SDR and sim
0013 reds all close
0014
0015 the sim cyan looked a bit weaker otherwise SRD was a bit greener cyan than HDR
0016 SDR and sim match but red neon is more pink and green neon a bit weaker than HDR
0017 SDR and sim weaker or less sat than HDR plus SDR has grey patch in sun (clipping?)
0018 HDR looks to have more sat
0019
0020
0021 HDR has a bit more sat
0022
0023
0024
0025 Red neon in HDR is pink in SDR and sim
0026 HDR has more sat
0027
0028
0029
0030 bananas and melon look weak in SDR and sim and more sat in HDR
0031 rather than SDR and sim looking weak HDR might be more sat
0032 rather than SDR and sim looking weak HDR might be more sat & Red neon different
0033 SDR and sim look grey (Sim more so) might be sat difference in blues
0034
0035 SDR and HDR match better than sim which looks grey or has magenta in blues
0036
0037
0038 neon seems to match better in this frame
0039
0040
0041 more green in HDR
0042 HDR has more sat
0043 blue in upper left sky of sim not as blue as others
0044 SDR and sim match HDR has more sat and neon name on building different (more orange than pink)
0045
0046 blues differ in all three
0047 SDR and sim look weak next to HDR (just not a lower dynamic scene)
0048 SDR and sim look weak next to HDR (just not a lower dynamic scene)
0049
0050
0051 is more contrast or luminance of HDR adding sat?
0052 is more contrast or luminance of HDR adding sat?
0053
0054
0055
0056 Red neon in signs of HDR goes pink in SDR and sim
0057 amazing close match, however screen seems shifted to purple
0058 all match with the same sat/contrast differences in skin
0059 all look orange and maybe not enough red for what has been called red Christmas, but they match
0060
0061
0062
0063 the red, blue green only work in HDR, note that the chart looks a bit weak in SDR and sim
0064 HDR looks too much sat, enough to not look real
0065 SDR and sim match but tail lights differ from HDR
0066
0067 I think the shirt is supposed to be powder blue which looks a bit magenta in HDR
0068
0069 immediately noticeable, the ferris wheel lights differ between HDR and SDR, sim
0070
0071 again the red neon
0072
0073
0074
0075
0076 again HDR is too much
==============================
Purpose - to check inverse IDT
A first node with ACES transform ODT v032
A second single node with ACES transform Input as inverse032 and Output as v032
checked for Rec.709, Rec.2100(709sim), and Rec2100(P3D65 1000nit)
Note that if the order of the nodes is reversed all fail without question.
Instead of listing a bunch of notes (available if desired) will just say that the inverse does not look to work well.
==========
Equipment:
Computer A - for HDR and SDR
AMD Threadripper 1950X on Asrock Fatal1ty X399
Windows 10 Pro 64-bit 21H2
AMD Radeon Pro WX 8200
BMD Decklink 4K Extreme 12G
LG C8PUA OLED TV calibrated & checked with Xrite i1Display Pro
Davinci Resolve 18.1.4 build 9
Computer B - for HDR and SDR
Apple Ipad Pro M2 12.9"
DaVince Resolve for iPad (latest)
Note: on the iPad for some reason files cannot be placed into the directory for ACES transforms so DCTL were used instead of ACES Transforms by commenting out the first line of code.
2 Likes
I opened a new pull request for @alexfry for CAM DRT v033, also available in my repo.
I spent some time looking and improving the SDR/HDR match. This version should have a match that is on par with v31, the last version that had a good match. By scaling the compression curve (aka. tonescaled J / original J) in non-linear fashion I was able to improve the match.
This version adds also a new custom chroma compression gamut. It can be adjusted inside the kernel parameters as xy coordinates. Previous version was using rec.709 for the compression but as agreed it is better to define our own.
EDIT: I closed the pull request and opened a new one (#16). I noticed I had left wrong gamut mapper parameters in the node. No other changes.
I think you may be expecting too much of the inverse. It does not aim (nor would it really be possible) to take ACES2065-1 values transformed through a forward DRT and revert them back to the original ACES2065-1 values.
Rather the intention and purpose of the inverse is to be able to take display-referred image data, either from existing display referred imagery, or from processing scene data through a different rendering, and invert that to a limited set of ACES2065-1 data such that when subsequently processed through the forward transform it lands back at the previous display-referred values.
TLDR: inverse followed by forward DRT should be (close to) a NoOp for input in the 0-1 range. Forward DRT followed by inverse is not expected to produce a NoOp for all input values.
While debugging my procedural Baselight DRT, I came across a slight discrepancy between the forward and inverse gamut compression.
The current gamut compressor uses a gamma approximation to the lower part of the gamut hull. This in turn uses a further approximation, which I came up with, to find the intersection between the compression vector and the curved boundary. This approximation does not land exactly on the boundary, but since our gamma curve is itself only an approximation anyway, and we are compressing rather than hard clipping at the boundary, I did not think it was an issue. However I have discovered that the use of the source JM value in the boundary finding gives a slightly different result when finding the boundary distance from the compressed colour for inverting than it did from the original colour for compressing.
I suspect the discrepancy is normally pretty small, but since our intent was to create a fully invertible gamut compression, it would be better if the normalisation distance used was identical in both directions.
If somebody with better mathematical skills than myself can find a way to find the exact intersection between a straight line and a power curve, that would be ideal. Otherwise I have been experimenting with a method which uses only the J-axis intersection value, not the pixel JM values, to calculate the normalising boundary distance. I will demonstrate that in tomorrow’s meeting.
The LUT repo has been updated with your new v033 work.
I’ve only had a brief moment to look at it.
But the 100nit → 540nit → 1000nit progression looks much cleaner than v032.
1 Like
If I’m following along correctly, then if k is rational, which in our case it should be, it is possible to turn 0 = ax^k - mx - c into a higher degree polynomial by change of variables e.g.:
I’m not sure in general that helps but there may be a special case that could be found.
1 Like
ChatGPT gave me not working formula for a simple Lift operation I was trying to build from nodes in Unreal shader editor 
It gave me this:
Output pixel value = (Input pixel value - lift) / (1 - lift)
And it says it is eqivalent to this (the one I googled and that I managed to make working in Unreal)
Output = ( input * ( 1 – lift ) ) + lift
Maybe it’s the same. It’s too complicated math for me and maybe I just made a mistake while building the first formula using nodes.
It also says that
Output = (lift * (1 - input)) + input
represents a color grading gamma operation, rather than a lift operation.
I’m really dumb at math, but I always thought Gamma is a power function, and it is not in this formula.
Comparing SDR and HDR using v033
rec709
rec2100(rec709sim)
rec2100(P3D65 540nit)
rec2100(P3D65 1000nit)
SDR and HDR match much better then with v032 and are fairly good for most frames.
For example 0070 is a much better match than with v032.
I checked these with SDR on computer A and HDR on computer B as well as HDR on computer A and SDR on computer B.
Some issues were observed.
In most frames the rec2100(rec709sim) is not as good a match to other HDR as is the SDR rec709. The rec709sim seems to be not working as expected.
0012 Hollywood sign still red in HDR and more orange in both sim and SDR
0016 neon red as above^
0025 neon red as above^
0015 blue looks more grey in the rec2100(rec709sim) then either SDR or the other HDR
0033 same^
0035 same^
0046 same^
0017 still looks like clipping in the sun in rec709
0060 reds still look more orange in all (or maybe this is right?)
0061 same^
0063 same^
==================
INVERSE CHECK
With node as last time IDT and ODT toggled on and off.
SDR & sim
All the inverse IDT looked to work with v033 for SDR.
Very little differences in scopes and nothing really noticed in the images including the CGI frames.
HDR
Problems with green screen and the CGI frames but much better over all with some small differences noticed only in the scopes.
1 Like
Thanks Jeffrey for testing. I would agree and say that blue and red are the weakest colors when it comes to SDR/HDR match. It is the same with every previous version as well, as far as I have observed.
That’t really interesting. I suspect, however that the root of such a high order polynomial may be non-trivial to find. We can certainly look into it, but it may well be that because the power curve is only an approximation of the gamut boundary, a simpler function which gives a reasonable approximation of the intersection may be fine.
I’m a dumb dumb with math, so not sure how much use I can be haha, Which model did you use in chatGPT? I asked GPT4 and got this response.
The ChatGPT screen-shot I posted is a small part of a longer answer to my second question, “What if the exponent is not an integer?” My first question got a similar answer to yours, with the quadratic as an example. But we know we can easily solve those (it’s used elsewhere in the DRT). The exponent in our case is ~1.15, which is what makes the solution tricky. ChatGPT gave similar suggestions in the rest of the answer, that I didn’t show, regarding Newton Raphson, etc.
1 Like
I opened a new pull request for @alexfry for CAM DRT v034, also available in my fork.
This version adds support for soft clamp to avoid clipping (enabled by default). In contrast to the soft clipping experiment I made earlier, which compressed both negative values and values above 1.0 in display linear, this version compresses only negative values in display linear. Values above 1.0 are hard clamped to 1.0, same as before. Inside the kernel parameters there’s clamp thr and clamp dist parameters to allow adjusting the compression threshold and distance.
This version also has a small refinement in SDR/HDR match. To my eye it’s now better than v031 was.
1 Like