I mentioned in previous meetings that I was looking into my alternate approach to finding the gamut boundary for gamut compression, to make the gamut compression invert more precisely, and why that seemed to be causing a brightness shift in near-achromatic pixels.
I believe I have found the reason. Even when the colour is well within the threshold, so no gamut compression should take place, the code is calculating a value v which is the ratio of the distance of the colour from the J-axis to the distance of the boundary from the J-axis. It then compresses v, which will be unchanged if it is below the threshold parameter for the gamut compression (which it obviously will be for near-achromatic values). The final JMcompressed value is found as the proportion v along the line from the J-axis intersection to the gamut boundary. In theory for colours well within gamut this should all cancel out, and return the original JM value. However, for near-achromatic colours, the value of v is very small, and it appears to be processed in DCTL with insufficient precision to get back to the original value when multiplied again.
A simple 1D analogy:
\space \space \space x_{in} = 1.1
\space \space \space d = 100
\space \space \space v = \frac{x_{in}}{d} = 0.011
But let’s say our system precision only allows us to store v to 2 decimal places, so it becomes 0.01
The compression function does nothing to v because it’s below the threshold, so it remains 0.01
\space \space \space x_{out}=v \times d = 0.01 \times 100 = 1.0
So because of precision, dividing by d and then multiplying by d has the net effect of changing 1.1 into 1.0 rather than leaving it unchanged. The reason for this is not immediately obvious when you say you are working to two decimal place precision.
I therefore added an extra line to my DCTL implementation so it bypasses the compression entirely if v is below the compression threshold. This seems to have solved the problem, and is implemented in DRT_v31_709_alt.dctl in the latest commit to my repo.
I realise of course that this is still based on v31, so is behind @priikone’s latest experiments. As I have said previously, I will bring the DCTL up to date when things stabilise a bit more.