A variation on Jed's RGB gamut mapper

In meeting #20 I demonstrated a variation on @jedsmith’s RGB gamut mapper. My thinking was that changing a single channel because it is out of gamut (negative) while leaving others unchanged is altering the RGB ratios in a way which might distort hue. So I have experimented with an approach which takes the compression ratio applied to the distance from achromatic (maxRGB) of the negative channel, and also moves the other channel closer to maxRGB by the same ratio, even though it doesn’t “need” moving.

An exaggerated example to make the point. In Jed’s method, because the distance of red to blue is less than 1, it gets left as it is. Only green is moved towards blue, in this case reducing the distance by 25%. The alternative method also moves red towards blue, reducing it’s distance by 25% , because that is the amount that green was moved by. Even if red had been slightly negative as well, it would still have been moved by 25%, rather than a compression ratio calculated from its own distance, because that is the ratio for the furthest away colour, i.e. green.

I built the original experiment in Nuke, and it uses only my own rebuild of the Reinhard compression method. But will try and incorporate it into a branch of Jed’s full implementation, with all curve options, and a DCTL version as well. But that will take longer.

For now, here is my Nuke script:
alt_compress.nk (37.1 KB)


Hey @nick!

Thanks for the experimentation. It’s an interesting idea compressing the maximum of the inverse rgb ratios instead of the rgb ratios independently. (By the way, the maximum of the inverse rgb ratios is identical to saturation in the IHLS colorspace that I mentioned a while back which actually helped me figure out this method) .

The “jedsmith” RGB gamut mapper does modify both negative color components if that rgb triplet falls in an “overlap area” - like an area where blue and green distances are being affected by the compression.

For example, here’s the old chromaticity diagram with rgb values of ACEScg that I’ve been using to visualize this problem. Here with no gamut compression applied.

And here with gamut compression applied:

Note both the R and G values are being pushed positive.

If we compare the effect on the same image between your tweak and the original algorithm, it’s interesting what is happening.

Here’s the unaffected image with out of gamut values outside of the ACEScg gamut triangle:

Here is the same image, gamut compressed with the original algorithm (same parameter settings as your example: threshold 0.2, max distances 0.2, reinhard):

And here’s the same image gamut compressed with your tweak:

We are adding more compression in the region between the primary vector and the secondary vector. That is, between the line going between the whitepoint and the blue primary, and the line going between the whitepoint and the yellow point, halfway between the blue and green primary, we are adding additional compression.

Here is just the green channel after gamut compression with the original method:

And with your tweaked method:

As you can see, areas along the green primary line are the same between both methods.

It might be interesting to look at the difference between the original and the gamut compressed images.

Here is with the original method:

And your tweaked gamut compression:

The hues are obviously inverted here, but it’s obvious there is more yellow between green and red. More magenta between blue and red. More cyan between green and blue.

To put it simply, there are more secondary colors between the primary colors.

There are also more noticeable mach bands which might be problematic.

Looking at some real images… it seems to emphasize secondary colors and de-emphasize primary colors. For example here’s one of @colorbycontrast’s images with original method gamut compression:

And with the tweaked method gamut compression:

The purple colors are more purple, and the boundaries between purple and blue are sharper. Maybe unnaturally sharp.

The same thing is visible in @Justin_Johnson’s diffraction grating images. Here’s a cherry-picked example with the original method gamut compression applied:

And with your tweaked method:
Note the hard boundary between purple and cyan.

Looking at the Red Xmass image, it certainly does look more pleasing out of the box with your tweaked method. But is this because of the method or a happy accident?

Interestingly, in the faces of the children illuminated by the red light, only the blue component is negative. The differences between the original and your tweak is in how much the blue channel is increased in intensity.

Here’s an example - the original image with no gamut compression:

The image with the “original” gamut compression method:

With your tweaked method:

And here’s another version using the original method, but instead of the parameter settings you are using (threshold 0.2, max distances 0.2), I’ve increased the green threshold from 0.2 to 0.36:

It’s not exactly the same but it is quite close when you look at the skin tones, and there weirdness in the tonal gradients of his lips.

Adjusting the threshold values independently per color component might be an important aspect of the parameterization beyond the abstract “how much of the core gamut to protect” concept which we’ve all been thinking about.

Currently this option is only available in the Nuke version of the tool because I could not figure out an elegant way to put that into the other DCC implementations. Three sliders to set the threshold doesn’t really appeal to me…

Anyway sorry for the giant wall of text, and thanks again @nick for the experimentation. It got me thinking about this again …


Also, one other note after watching the last meeting regarding casual shirt guy in the Blue Bar - The artifacts that you pointed out on the back of his shirt are caused by the display rendering transform, not the gamut compression.

Here is the gamut compressed image you showed (the “jedsmith” gamut compression algorithm, threshold 0.2, max distances 0.2), with the ACES Rec.709 display rendering transform applied (using the ACES 1.2 OCIO config transform here)

The visible artifacts here are caused by color values being out of display gamut after the XYZ -> Rec.709 + hard clip transformation in ACESlib.OutputTransforms.ctl, lines 144 to 174. Interestingly if you chuck a gamut compression in there before the clip it looks better :wink:.

Here’s the same image using the Red IPP2 display transform from your shared OCIO config. No artifacts!

I guess this illustrates just how important the display transform component of these types of evaluations are…


Thanks @jedsmith for the thorough investigation. That’s a very good point you make about the artefacts being related to the display transform. It emphasises that the gamut mapper is not creating the artefacts on the back of the jacket, but rather what it does to the image is not preventing them occurring under the current Output Transform.

I have made a first pass at creating versions of the alternate Gamut Compressor in DCTL, BlinkScript and Matchbox. I’ve tested in Resolve 16.2.3 under CUDA, OpenCL and Metal on Mac and CUDA on Windows. The Matchbox shader has been tested in Flame 2021.0.1 and Baselight Student 5.2.13115.

I think that I need to add some additional trapping for NaN / inf, but I wanted to make these available for people to try ASAP. Bugs can be fixed later. Or the experiment can be abandoned if people feel it is not a path worth going down.

GamutCompress_0_3_alt.dctl (8.6 KB)
alt_compress.nk (38.0 KB)
GamutCompress_alt_matchbox.zip (4.6KB)

Note: The Nuke script is an updated one with the BlinkScript node modified to have an “alternate method” check-box. But the simple “pure Nuke” version remains as I showed in the meeting.



@nick, what an elegant piece you wrote there, nice RGB Ratio Kung Fu :-).
You map max and min on 0 and 1 so you can scale the middle component, very smart.

Some initial thoughts:

I guess the most noticeable thing is that your approach does not drift towards the primaries. In @jedsmith work, the compression function brings the two non-max components closer to each other, which is a hue-skew towards the primary defined by max(RGB).

The primaries location are the points furthest away from the white point, for each hue. This means compressing towards them is a “space-efficient” way of squeezing out of gamut colours in gamut. Also, the distance from the primaries to the white point is different for each hue. I found that the hue preserving gamut mapping is a bit harder to set up, as there is no way to bias the gamut mapping directions (like with the CMY weights).

Then on the other side, you have a hue-preserving gamut compression.
We need to find out for which use-cases that would be a beneficial property.

I will play with it more next week.

Great job.


Here are a new set of downloads for the alternate gamut compressor. The potential divide by zero error is fixed, and all three (C/M/Y) sliders are now used, where the initial experiment only used the first one. It now defaults to alternate mode, but can be switched back to @jedsmith’s original version by un-checking “alternate method”.

GamutCompress_alt_v2.dctl (8.9 KB)
GamutCompress_alt_blinkv_2.nk (26.8 KB)

Downloads withdrawn due to a bug please use Jed’s version below


I created v0.5 release in the gamut-compress repository.

  • Add “hexagonal” checkbox to toggle between Nick’s method and the original. (if anyone has a good suggestion for a better name I’d be open to it!)
  • Separate threshold sliders in all implementations of the tool.
  • Add ACEScc working colorspace option to the matchbox and dctl versions of the tool.

Further testing with the primaries+secondaries hexagonal method reveals really positive and promising results. One of the things that was causing the poor results in my numerous screenshots above was the equal max distance parameter values of 0.2. The max distance parameters are still important and have a big impact on the resulting hue. With the hexagonal method, there is much better handling of secondary colors, and as @daniele mentioned, less hue distortion near the primaries (red out of gamut colors getting compressed towards the line between the whitepoint and the red primary for example).

1 Like

Just a note to say that people should use @jedsmith’s implementation of my method for testing. My v2 introduces the possibility of creating colour discontinuities, which is obviously a bad thing!

That’s a good point, and I think that the need to compress more when further from primaries if it’s in a straight line is the reason for the ‘pointy bits’ in the plots @matthias.scharfenber showed in last night’s meeting when ‘hexagonal’ mode is enabled.

I did some experimenting today, taking the direction of the compression vector from the hexagonal version, and the magnitude from the original. It removes the points, but the resulting shape is rather peculiar…

Additionally inversion does not currently work.

I think it was an interesting experiment, but is probably a dead end.

I have today also discovered that the inversion in the current implementations does not work if “hexagonal” mode is enabled. Inversion is simple if (as in my animation at the top of the thread) the same compression parameters are used for all three channels. However, if (as is the current default) there are different limits per channel, it is possible for a channel with a larger distance to end up being more compressed, and therefore end up with a smaller compressed distance than a channel which had a smaller distance originally, but ends up less compressed. This then messes up the inversion of the sat calculation, as that is the max of the two channels which are not equal to ach, and which one is max can be flipped in this situation.

I’m sure an inversion is possible, but it will involve several extra calculations and compare operations. Is it worth it? I’m thinking that the hexagonal option is not worth pursuing. What do others think?

I have an experimental branch of the Python implementation which fixes the hexagonal inversion issue. It does it in a rather clunky way, so I want to think more about it, and get feedback from the group as to whether it is even necessary, before trying to port it to the other implementations.

This branch also removes the shadow roll off, as it was causing issues as described here. Those issues made diagnosing the inversion difficult. The necessity of the shadow roll-off is still up for discussion.