# Gamut Mapping Part 2: Getting to the Display

I don’t think it is incorrect quite the opposite, but the way they model our perception is maybe not appropriate for our needs. Also, given their design simplicity, they cannot be good at predicting lightness, chroma and hue at same time. Their is so much that a 3x3 matrix and a power-like function can do! The other space that might be also interesting to look at is IgTgPg, even though it does not produce great gradients!

I will render some hue stripes later for all of them.

Don’t get me wrong, I think this is all great, but also keep in mind that those colour space were never designed for the task at hand.

Yes, this was my point!

Disclaimer: The following is not an approach I would suggest be used for producing good looking images, but merely an interesting and informative exercise to better understand the problem at hand.

Back to Basics
The last several days I’ve been thinking about the highlight desaturation problem. With display rendering there are a lot of intertwined problem areas, and it’s very easy (at least for me) to get confused. Like earlier when I was thinking about gamut compression and highlight desaturation as one problem. They are two different problems. Related, but not the same.

So how can we simplify and isolate these problem areas? We could ignore “out of display gamut” issues. We could ignore tonemapping or intensity scaling issues.

NaiveDisplayTransform
In an effort to simplify and focus on highlight compression and desaturation, I made a nuke node called NaiveDisplayTransform. As the name suggests, it undertakes a very naive approach to mapping scene-linear to display-linear.

• Maps an input range in scene-linear from `linear_start` to `linear_end`, defined in stops above and below middle gray, to an output range in display linear from 0 to `display linear end`.
• From `display linear end` to 1.0, apply a compression curve to the scene linear values. The compression curve compresses infinity to a value of `limit` stops above `linear end`. In other words: if limit is 1, all highlight values from `linear end` to infinity will be compressed into a range of 1 stop. The bigger `limit` is, the more range there will be for the compressed highlight values.
• Where highlight values are compressed, desaturate.

The simplest possible display transform would be a linear to linear mapping. We take a range of scene-linear values and remap them to a range of display-linear values. We apply the inverse EOTF. The scene linear values are displayed directly on the display, up until the values clip. For the range that can be displayed, (ignoring spectral emission differences, calibration, and other complications) there should be a 1 to 1 correspondence between scene luminance and display luminance.

Since pictures are easier to understand than words (at least for me), here’s a video demonstration of the concept and the tool.

When Channels Clip
With a simple 1 to 1 mapping, we can focus on how highlight values behave as they approach 100% display emission.

With no highlight compression and no desaturation, hue shifts are introduced for non achromatic colors, because as one component clips, the other components continue to increase. So we need some way to “handle” these components as they increase in brightness, to remove hue shifts as the components approach clipping. To do this we can move the components toward the achromatic axis as their luminance increases.

Technical Approach
Amidst an extended conversation with @Troy_James_Sobotka (thanks for your patience with my stupid questions), I got to thinking about the Inverse RGB Ratios approach that we ended up using in the Gamut Mapping VWG. In that approach we were using the inverse rgb ratios to push out of gamut values back towards the achromatic axis.

``````inverse_rgb_ratio = max(r,g,b) - rgb
``````

If we instead used a constant value that we wanted to “collapse” our components towards, we could do

``````lin_max = 4.0
inverse_rgb_ratio = lin_max - rgb
``````

If we add `inverse_rgb_ratio` to rgb, we will get `lin_max` everywhere. But if we modulate `inverse_rgb_ratio` by some factor which describes how much we want to desaturate, then we get a simple and effective method of highlight desaturation.

The best way I’ve found so far (and I think there’s better approaches), is to modulate the `inverse_rgb_ratio` by the complement of the compression factor. When we use the norm to do the highlight compression, we do

``````norm = max(r,g,b)
intensity_scale = compress(norm) / norm
scaled_rgb = intensity_scale * rgb
compression_factor = 1 - intensity_scale
``````

`intensity_scale` here is kindof like the derivative of the curve: it represents how much the compress function is altering the norm.

Then we can do

``````lin_max = 4.0
inverse_rgb_ratio = lin_max - c
desaturation_factor = inverse_rgb_ratio * compression_factor
desat_rgb = rgb + desaturation_factor
``````

Considering how stupid and simple this approach is it actually doesn’t look half bad in my aesthetic opinion.

And finally here’s the NaiveDisplayTransform node if you want to play around with it. No blinkscript so it should work fine in Nuke Non-Commercial.

EDIT - since the above link points to a gist that has been updated, here is the original nuke script as it existed at the time of this post: NaiveDisplayTransform_v01.nk (18.3 KB)

Next step for me is seeing how this same approach might be applied with a more traditional sigmoidal curve for input range compression.

Just wanted to share what I’ve been up to in case it’s useful for anyone here.

4 Likes

Nice stuff !

I have been playing with the Nuke script this week and it has given me interesting results. I think it is great to go back to basics in a way (at least for me !) in order to fully understand what we are dealing with.

I have done a quick image of the tool’s options/parameters if anyone is interested. It helped me to grasp better these concepts.

I am looking forward to future updates of the tool. Let me know if my “mockup” is wrong or incorrect, I’d be happy to update it.

Chris

1 Like

Thanks @ChrisBrejon! Now that you’ve made this excellent diagram explaining everything, I reworked the parameters a bit to hopefully make it more clear what is going on. I realized after reading my post again with a fresh brain that there were some things that could be made more clear and simplified.

I reworked the parameters so that the naming is more consistent, and so that the linear value that is calculated is displayed right below the value you are adjusting.

Everything functions the same way except for the `limit` parameter, which I’ve changed the name of to `compression` and altered how it works. Now instead of mapping infinity to the value that you specify in stops above `lin_white`, you specify a value in stops above `lin_white`, and that value is compressed to display maximum. In other words, `compression` calculates the `max` value, which is the maximum scene-linear value that will be represented on the display.

I’ve also exposed the strength parameter, so you can adjust the slope of the compression curve. It seems like you need control over this to get the best results, depending on the `dwhite` and `compression` value you have set.

EDIT - Here is the nuke script described in the above screenshot
NaiveDisplayTransform_v02.nk (6.9 KB)

I also made some further usability improvements and additional parameters, which I did not make a post about. Here is that updated version:
NaiveDisplayTransform_v03.nk (7.6 KB)

EOTF should be EOTF^{-1} or Inverse\ EOTF and we should try to remove gamma if possible too!

3 Likes

Thanks @Thomas_Mansencal ! Indeed, that’d be much better this way ! I’ll update the sketch asap.

Cool @jedsmith ! I’ll give it a try next week ! Great update and video explanation. Thanks for sharing !

Jed,

Thanks so much for the work and thoughts. There’s a lot here to react to. I’ll just throw out a few points to consider.

• I don’t know if I’d characterize highlight desaturation as a problem. Sometimes it’s the effect you want, and sometimes it isn’t.

• I think there’s value in recognizing, either conceptually or in practice, that the rendering transform, at some point, should yield display colorimetry and turning that display colorimetry into a signal is a process that’s pretty straight forward and objective (e.g. convert to display primaries, encoded with inverse EOTF, etc). The only wrinkle is gamut mapping. We’ve talked about this We’ve talked about the separation of rendering and creation of the display encoding a bunch in the past and it’s fine to separate that out but it’s not really part of the rendering discussion per-say. We discussed the concept in this document. http://j.mp/TB-2014-013.

• Displaying linear scene data on a display, within the limits of that display’s capability, is a helpful tool at times. With the ACES 1.0 work we did that a lot. I also suggested doing that during the gamut mapping work to help visualize the ACES data. Generally speaking, to make a reasonable reproduction on an output medium you’re going to need to compensate for viewing flare and the surround associated with the display environment (among other things). But those two usually end up increasing the slope of the tone scale. I’d highly recommend Chapter 5 of Digital Color Management: Encoding Solutions Second Edition which does the topic way more justice than I ever could.

3 Likes

Hey Alex, thanks for the thoughts!

Thank you! I’m in this to learn, and I appreciate the reference. I’ll take a look at this. As I mentioned in my disclaimer above, this experiment is purely to better understand the problem at hand, not something I am putting forward as something to be considered as an aesthetically pleasing display rendering.

My Apologies for the imprecise terminology. As I’ve mentioned in the past, I could more accurately be referred to as a “Color Pragmatist” than a “Color Scientist”.

The problem that I’m attempting to understand in my ramblings above could perhaps be more precisely expressed.

What I’m trying to understand is what happens to color when approaching and passing the “top end” or max luminance of a display-referred gamut boundary. That is, for an rgb triplet which contains a color that is too bright to be represented on a display device, what happens to hue when one channel clips to display maximum and other channels do not. These unnatural renderings of hue in the upper portions of the luminance range is why I think gamut reduction as luminance approaches display maximum is critically important to rendering a good looking picture, rather than an aesthetic preference. The example photos that you posted here show the issue pretty clearly.

Of course the implicit assumption in all of this is that we would want a chromaticity preserving tonescale rather than a per-channel rgb approach which applies this luminance-gamut limiting as a byproduct.

Can you elaborate on why gamut mapping should not be a part of the rendering discussion? Surely there should be some type of handling for colorimetry that can not be reproduced on the display device in the default display rendering transform? I’m sure it’s something silly I didn’t think of, but like I said I’m in this to learn

2 Likes

Hi Jed,

Hey @TooDee - Sorry I didn’t specify in my post: it’s updated at the same link,

1 Like

Totally understand … thank you for playing with this and sharing what you’ve learned.

I think that’s the part I’m reacting to really. I don’t know if we do or don’t. A chromaticity preserving tone scale sounds great, makes great plots, looks great on some images and absolutely horrible on others. We spent a ton of time in the ACES 1.0 development process looking at a chromaticity preserving tone scale it it was like an onion. We’d peel back layers and just run into more and more issues. The models kept getting more complex and eventually we pulled the plug. I think it’s worth looking at again but I’m not sure we want to make it a hard requirement.

This is just the way my brain splits up the problem but going from scene-referred colorimetry to the colorimetry as we want the image to appear on the display is the rendering. For the sake of argument let’s call that output XYZ. There’s a separate, and relatively easy, task that needs to occur to take those output XYZ values and turn them into a set of code values. When those code values drive the display in question measurements of the light coming from the display should match the rendering output XYZ values.

Obviously, depending on the details of the rendering, the output XYZ values may exceed the capabilities of what the display can actual show. If that happens choices need to be made in the signal generation stage. With this sort of conceptual split though, you can do interesting things. For instance, you can make the rendering create output XYZ values for a limited dynamic range and a small set of primaries (e.g. rec709) and then generate the corresponding code values to correctly display those output XYZ values on a display with much higher dynamic range and a larger set of primaries. This effectively lets you use an HDR monitor as a SDR monitor without changing the monitor calibration. @sdyer and I do this all the time with our BVM-X300.

1 Like

The simple fact it would be such a departure to the current rendering makes it a no-go as a default for me. It does not mean we should not provide a way to be able to do it though, the advantage is that it could be introduced alongside and let people test and migrate their workflows to a hue-preserving one.

100% agreeing for the gamut-mapping point!

Cheers,

Thomas

Just to complement the hue-preserving discussion, I remembered the presentation Alex Fry (EA) gave at the GDC 2017: https://www.gdcvault.com/play/1024466/High-Dynamic-Range-Color-Grading

2 Likes

Sorry I may not have respond to this directly. I wasn’t trying to say gamut mapping wasn’t part of the rendering but rather the conversion from output Colorimetry to display code values. If the rendering produces values outside the displays capability obviously gamut mapping is needed. Probably better to make the rendering not produce out of gamut values to begin with though.

Thanks for clarifying. As is so common in conversations about color, differently understood terminology seems to be the source of my misunderstanding here. As your diagram shows, I believe you are thinking about “Rendering” as a separate step from the transform to get to display colorimetry, where I was thinking of it as the whole process to get from scene-linear to display.

I do wonder though - is it really possible to separate rendering from transformation to display colorimetry?

Isn’t this impossible? - (This may be another stupid question - like I said, color pragmatist)

What if we need to create a rendering for Rec.2020 and for Rec.709? Are we going to use the same gamut mapping approach? Would we limit the colors to the smaller one? Or would there be two different approaches specialized for each display gamut?

Certainly we could conceptually move the gamut mapping problem into a box between “Output XYZ” and “Display Code Values” but it seems to me like this would still need to be a specialized process for each output display gamut?

Please let me know if I’m missing something here, I am genuinely curious.

There has been a very interesting conversation on slack and I thought it would be worth doing a summary on acescentral.

my only point is that removing hue shifts as a default is not desirable because of the arguments Alex Forsythe brought up and the simple fact it would change the look massively. […] Reminded me Alex Fry GDC presentation from 2017 where he explained that he was adamant about removing them but it did not worked for all cases, quite the opposite!

Which @Troy_James_Sobotka replied :

Optionally, I think the point is being missed.

1. The display can’t represent the colour of the flame, so it skews.
2. It is possible, if one were so inclined, to add a saturation adjustment to return more of the “yellowish” flame.
3. It is impossible to remove the saturation surface of the image once the skews are introduced.
[…] But again, the crux of this discussion is essentially about why bother with colour science at all if it never comes out of the display?

The conservation followed like this with @Thomas_Mansencal explaining :

Without entering into subjective discussions, changing something like the default tonescale contrast has much less impact than making the tonescale hue preserving. So making the latter change a default in a new ACES rendering transform has much deeper consequences, so I don’t think it is a good idea, it should be optional. If you have built a ton of assets under the current rendering transform and you end up in a situation where the new one forces you to rebalance all of them, it is not a great change. […] Not saying there is no problem, just saying that the medecine should not be the nuclear option.

If the skew is part of the Display Rendering it needs to do the same skew for all deliverable. In fact your cg assets will look completely different in HDR under current transforms. […] Well the starting point should be same appearance , I guess.

Which Thomas replied :

They will (look different) and it is acknowledged, entirely depends on the deliverables. And the client might also either be fine with a different look on HDR but even request for it. […] Just look at the Haywire trailer by Steven Soderbergh… Not saying that I like it, but the client pays the bills here and he is certainly pushing for that look. I don’t know if the appearance should be the same with vastly different class of displays: are clients expecting that the appearance of a 100nits, 10000nits and the real world appearance are the same? It is a fundamental question to which I have no answers. Say I’m paying extra bucks to go to HDR cinema, what is the point to get there if things look the same than SDR exhibition ? That is the type of questions producers ask themselves.

Then Daniele explained :

I am not saying HDR and SDR should be identical. But a yellow candle in SDR should not be orange in HDR I guess. At least I know many colourists which are confused when that happens.

To which Thomas replied :

I certainly could see that happening but all your pyro effects turning red/white under a hue-preserving rendering transform is equally bad, and it does not affect a single person but hundreds of people. All of sudden, all your Lookdev, FX, Lighting & Rendering teams are affected. Sometimes, broken by force of adoption is what is correct, but we are entering philosophical discussions here. And again, what feels broken for you might be a feature or the expected behaviour for others. […] I can certainly point at the dangers of changing the rendering transform in such a dramatic way that it preserves hues. […] Changes to a a large system like ACES must be incremental, you just can’t modify something as critical as the RRT without triggering a storm, instead you should give people the option to change their workflow, at their will and gradually. Happy to be corrected but it seems like the only sane thing to do to avoid having an angry mob knocking at the Academy’s door!

Jed entered the conversation :

2.0 seems like a good version to make a change to the default display rendering transform. I’m not sure there would be an angry mob if it looked better than the current version. I think a lot there are a lot of people especially in vfx who use aces because it is easy and off the shelf, who think that it is a “neutral” view transform, and are trusting it to represent their asset work faithfully, and they do not realise what is happening to the color that they are so careful to create faithfully once it goes through the view transform to the display. Certainly a chromaticity preserving rendering method is not the easiest path forward, but I do not think an approach should be discarded because it is difficult or inconvenient. As professional image makers we should be able to make intentional decisions about the appearance of color, to have control over what is happening. I think there is benefit in the approach of trying to pull apart the reasons why certain things are happening, why we are seeing certain artifacts. It is very difficult to break down problems like this because there are so many factors at play. But I think if we try to isolate problem areas maybe there are solutions we haven’t thought of.

And Daniele suggested :

You can make a LMT to match ACES 1.0?

@nick also shared his thoughts :

If we offer a few options out of the box, then I think the default should feel close to what we have now, to keep the people who already use ACES comfortable. But we could have a hue preserving alternate, and maybe people would gradually move over. Then we can deprecate it in 3.0.

A big question brough up by Thomas and Troy was about objectivity :

The problem is that looking better is highly subjective and any large DRT change has potentially a large cost.

I can’t help but find this “highly subjective” line of discourse completely quite disturbing. What’s our ground truth? We have a working space. We have values. We are not displaying them. At all. Not even remotely.

Interestingly enough it looks like the famous K1S1 DRT is not hue-preserving.

Then @sdyer entered the conversation :

[…] all options are on the table - if they meet the requirements. Which we are trying to define. everyone here seems to be acting like decisions have already been made. Can we focus on defining absolute requirements and then maybe even things that “would be nice to have” (if we can get them without breaking other stuff)?

We then agreed that we would need to see examples of chromaticity preserving tone scale sounds great, makes great plots, looks great on some images and absolutely horrible on others to better understand what happened for the creation of ACES 1.0.

I also tried to give my tuppence :

Keeping skews for a major release doesn’t sound right to me. For a major release, we should not be shy of changing stuff. This is the moment to do things “right” or “better”… Not 3.0. By definition, a major change means stuff will change and eventually break.

Which Scott replied to :

I can tell you I am 100% in favor of changing stuff, even drastically, IF it fixes the things we consider broken and still delivers on the other things we want it to keep doing. The requirements define the solution. so figure out what it must do and then we can focus our efforts on making something that does. We can’t sell a big change jsut for the sake of changing things. But if we can show the many ways on which it improves known issues with v1, we will have a lot easier time convincing the stalwarts to change and the holdouts to revisit it.

Finally Daniele shared some thoughts :

I still have difficulties to accept the “cg assets” argument. If the assets are so fragile to the DRT in use how do they survive a normal grading session. All the cg and VFX elements I got my hands on work very well with different DRTs and looks. They look different, but so does all the live action too. […] In a way a cleaner DRT should actually help to produce neutral assets.

Final words from thomas about CG issues :

Not all assets are equally fragile, pyro elements certainly would be. You have a whole class of people using pre-rendered texture sheets and such in games. And they tweak the look through the DRT directly.

I have tried to reproduce this conversation in the most faithful way. If you think I haven’t done a good job, please let me know and I will modify/delete the post. But I thought it would be important to share with everyone on acescentral.

• Should the new Output Transforms be hue-preserving ?
• Should there be a default behaviour and then an optional one ?
• Do we want a smooth transition or a fresh start ?
• Should we consult (again) with studios/people using ACES to see what they think ?

Regards,
Chris

4 Likes

Thanks @ChrisBrejon,

To complement the CG assets point, I made an emphasis on pyros, but it can be extended to any emissive source generally or assets with high-reflectance or that we are super picky about, e.g. CG skin.

As support to what could be the widescale effect, here is an old public thread that highlights the type of challenges that a change of DRT can induce.

The worst that could happen would be people trying again to apply the inverse new RRT with some custom massaging to maintain the previous look. Noteworthy, a significant portion of the time I spend on ACEScentral has been either explaining to people the ACES look compared to their “Old sRGB” one BUT also discouraging, sometimes with friction, their attempts at maintaining it by applying the inverse RRT on their textures.

I have been fortunate enough to work in a few studios and experienced DRT change from the first or second rows and, no, it is not as simple as flicking a switch.

Cheers,

Thomas

1 Like

@Thomas_Mansencal I have seen the effects of DRT change on cg asset look development. I agree that it is not a simple or well-understood problem, and it can cause serious headaches if the workflow is not understood and planned properly from start to finish.

My perspective here will be from feature film VFX - and from my personal experiences.

A conflict often arises in VFX studios when there is a desire for a consistent internal DRT in cg asset look development. An internal DRT has benefits for workflow, re-usability, and makes things easier for lookdev and texture. It can help a lot of things.

However, on different shows, the DRT from the client DI house is very rarely consistent. Every show has a customized display rendering transform.

At some point the cg asset has to be composited into a shot and sent to the client. If significant dialing of lookdev has happened under an internal DRT, and that asset gets rendered and comped there is significant danger for the appearance to change significantly - especially on more saturated colors like pyro, and more sensitive objects like human skin, as you mention.

So my question is: if a studio is using an internal DRT for cg asset work, don’t they already face the problem you mention? And as @daniele pointed out, wouldn’t a more chromaticity-accurate display rendering transform actually help solve some of these issues?

In the past, I have suggested workflows where initial cg asset work happens with the internal DRT, but at a certain point, evaluation of that work should happen under the show DRT, to avoid a big surprise when the work goes into comp.

Maybe cg asset look development work should even be checked under multiple DRTs in order to verify that assumptions are correct: an internal DRT, the show DRT, and a simple linear-to-linear display as you were talking about in your earlier post.

2 Likes