Re-Framing and Clarifying the Goals of an Output Transform

Hm, I’m not sure I’d agree. Well, I totally agree that

however I’d very much disagree that

I think @Troy_James_Sobotka put it quite well with this quote

[Typically], for non spectral based stimulus, the peak brightness range will rest along the achromatic axis of a medium. Chroma laden primaries or paints will almost always achieve a lower “brightness” at maximal emission / reflectance of stimulus.

In terms of the Monet, the sun is not white and yet it appears bright, surely… in reference to the rest of the image, as @Derek said. Yet, it doesn’t appear as bright as the sun would have appeared had we been standing next to him as he painted it. He’s made the decision that this isn’t important for him – he cares more about evoking the emotion that the fully chroma laden color does than about making our eye see the sun as brighter. There’s nothing (if we get it right) preventing someone from making the same choice even with a DRT that does use a ‘path to white’ – if the artist chooses to make the same tradeoff, there’s nothing stopping them from grading with a Look that severely caps the luminance of the sun down to a level that will be able to be directly displayed on their reference display (in which case the DRT shouldn’t affect its chroma, unless later retargeted on a less capable display).

Actually, this segues into a topic that I’ve been trying to wrap my head around enough to post about it in a comprehensible way, which is the topic of how to evaluate transforms.

Something that I think has been a little bit looked over as of yet has been the casual nature by which we’ve evaluated the proposed methods… basically it’s just been a matter of taking some raw scene-referred footage and passing it through and then evaluating the subjective/preceived ‘beauty’ of the result. I think that this is actually quite an ill-formed method of comparison/evaluation though. Troy posted a tweet thread today that I’ll ‘unroll’ here as it directly relates and finally connected the dots to make this fully comprehended in my brain:

When we have a stimulus encoded in data, no matter what we do, we are going to have that stimulus rendered in a medium like a display. We have no options; it will always come out as something. The idea that we can somehow ignore the limitations of the medium is what yields broken looking imagery. We must, at all times, start with the medium and work backwards to form an image. Why? Because again, whether we like it or not, every single stimulus encoded piece of data will be forced out as something. Either we can control that, and form it into an image, or we can ignore it and it will be formed into an image for us. This is not optional.

What does this mean? Well, it means that in evaluation we need a “ground truth” which is itself a fully formed image… i.e. an image which we don’t feed through our DRT but to which we can compare the fully-formed image output of the DRT. I’m honestly not sure of the best way to create this ground truth reference, though, which is an important piece of the puzzle. But the point, as said by Troy, is that this ‘reference formation process’ is a non-optional piece of the picture. By ignoring it, as we are now, we’re just leaving it up to chance, but it’s still ‘happening’–“doing nothing” is still a rendering transform, just one that doesn’t have a well defined output… so we should instead actually think about and actively form that process instead.

2 Likes

Hey Gray,

The Helmholtz–Kohlrausch and Hunt effects and most especially the former, are defining what I said earlier:

Cheers,

Thomas

Hey @gray,

we discussed these terms in this thread. There is some very good info from Cinematic Color 2 and Wikipedia if you hadn´t a look already.

I agree on these two points (thanks for writing them so clearly !) :

  • There’s nothing preventing someone from making the same choice (than Monet) even with a DRT that does use a ‘path to white’.
  • Something that I think has been a little bit looked over as of yet has been the casual nature by which we’ve evaluated the proposed methods…

I know that Daniele has suggested several times that the Academy could shoot a little scene (with like several iconic objects in it such as Macbeth chart…) and evaluate the scene and its display rendering at the “same time”. I cannot think of a better method to be honest. I have shared many CG examples to evaluate the DRT but yeah, that does not make it an “objective” test neither. Even if with CG we remove any “secret sauce” or clipping from a “real” camera ?

The “path-to-maximal-brightness” seems like such a fundamental mechanics when it comes to “tonality” :

If you keep the chroma yeah you don´t see the tonality of it anymore. If you can’t tell light from dark anymore, the image doesn’t look as natural anymore, right?

This is an interesting quote from Mathias in the last VWG meeting. I was pretty pleased to hear it since it directly connects with the “loss of tonality” that a few folks have been describing on this forum. I thought it was worth pointing it out.

Chris

1 Like

Sure, I agree (or I guess I should say, I know of that effect given it’s not exactly an opinion ;p)… I guess there’s some nuance here, though, in that I think we can also agree that if we are at a max luminance high colourfulness color on a particular display and then we create a gradient in the center from that high chroma version of a color to a less chroma-laden version of it (i.e. towards white), that we create the sensation of increasing brightness towards that white. No?

Yes, which goes back to something I was saying a while ago: if your display has a high peak luminance, e.g. 1500+ nits , you do not really need to desaturate anything to produce a red or a blue that appear bright. It is hard to convey with a photograph but your lightsabers appear damn bright on my screen (or LED Wall) and without desaturation or anything:

Could their core be white to make them more light saber-like sure, my point is white does not necessarily means bright. Using a DRT that implements path-to-white, I actually find the saturated light sabers to appear brighter, especially the red one that really pops!

Sure, that would be akin to modelling glare which is a visual cue for bright sources:

Which rows are brighter? This certainly goes back to the importance of the geometry of the stimuli and their spatial relationships. The DRT cannot add glare if it is not present in the image, it can shape it to a degree though.

Cheers,

Thomas

1 Like

Another quote from last meeting from @Troy_James_Sobotka is also worth bringing up:

So it makes sense to sculpt the curve that can hit the corners and characteristically curve inward from the corners.

And @jedsmith’s great post about how chroma compression works in OpenDRT. I still think this whole topic deserves a dedicated thread of its own. This “sculpting” (ie. engineering the shape) is critical, I think.

Like this post maybe ?

Update : Sorry I have written this in a rush and should have given more context. I have recently bought a laptop with an OLED HDR Display. The “reds” look incredibly bright to me, I was very surprised by that. So when I was shared this tweet and thought of this thread, I thought it would be worth mentioning.

Update : Ok. I can only imagine how bright it goes, since my display “only” goes to 500nits and my eyes are already burning. These HDRs display gamut volume can still be represented as a cube, right ? What happens to the red Light Sabers values if they don´t go to white ? Do they collapse on the gamut boundary ?

Chris

Sure, right, I agree with this statement alone for sure. If the “golden reference” is this very bright red image, which your display is capable of producing, then there’s no need for any desaturation at all and indeed the DRT shouldn’t desaturate. To quote myself (;p),

However, if we want to capture the same sensation when displaying the image on a standard SDR display which is simply not capable of that, then we need a well defined transform into something that the SDR display is capable of.

Or, indeed, if that rendering was changed so that its sabers were even brighter, we still need a well-defined transform into something that the 1500 nit display can show without leaving it up to chance. i.e. to prevent

But for every display. Hence what @Troy_James_Sobotka has been hammering home,

… whether we like it or not, every single stimulus encoded piece of data will be forced out as something. Either we can control that, and form it into an image, or we can ignore it and it will be formed into an image for us. This is not optional.

Even on a very high dynamic range display, there’s still some possible stimulus that may be encoded in scene linear values which will not be displayable (in fact it’s still quite easy to do so). So, we still need a well defined mapping for every value into the displayable range/gamut.

1 Like

LOL, you’re not kidding! My display goes to 1600 nits and I had a serious migraine after a few hours. Not fun. Raises questions too for me about whether or not HDR really is the “perfect” display. I know you’re supposed to suffer for art, but is the audience supposed to suffer too? :wink:

As hard as you try, no amount of path-to-white will help producing the same sensation. It is simply not possible and if so, we would not have HDR displays in the first place. What is possible to a limited degree is to change the viewing conditions, hoping that for example the HDR content was seen with a brighter surround so that you can reproduce the content on a display with a lower peak luminance in a darker surround.

Certainly but isn’t it the objective of the DRT, i.e. mapping an unbounded domain to a bounded range while reproducing the scene faithfully or at a bare minimum, pleasingly?

Both adjectives carry varying degrees of subjectivity, maybe the underlying question of the thread is how could we make that process objective?

Ha! :slight_smile: This is contextual, put your display outside and it will look like very disappointing! :wink: The content should be mapped appropriately as a function of display capabilities AND surround, i.e. viewing conditions.

1 Like

Well it’s not possible to recreate the exact same sensation, but the goal should be to create the same sensation as much as we can given the limitations. And path-to-white is quite a proven go-to strategy for doing this among many subtractive media throughout the past centuries–drawing, painting, and film.

Yes, exactly :slight_smile: this is my point

1 Like

Agreed, I don’t think anyone is challenging that idea, however, given that:

  • The dynamic range of drawing, painting, paper or SDR, e.g. 1000:1, is orders of magnitude smaller compared to HDR, e.g. 10000000:1.
  • The gamut of HDR being wider than SDR.
  • The quasi mutual exclusivity between being able to reach full display gamut volume and having a pleasing path-to-white transformation.

I’m mildly annoyed by the idea of a systematic blanket path-to-white transformation because it removes a lot of creative control in fixed system like ACES. It is important to know where you are coming from in order to know where you go and I’m confident that SDR will disappear, the same way CRT or PAL and NTSC did. With that in mind, and the profusion of highly chromatic stimuli IRL that do not cause HVS bleaching, I find it backward to systematically apply something that inhibits the potential of our new display technologies.

Something great about @jedsmith’s Open DRT is the fact you can tune the chroma reduction relatively easily. For me it always come back to creative control, everything should be simple but complex stuff should be possible. To give an example, I rolled a custom DRT for a show last year because our chroma reduction was affecting too much some yellow light fixtures and their reproduction on the LED Wall / Camera. I had to punch a hole for that particular hue angle so that the yellow hit display maximum capabilities while preserving the appearance of everything else. You obviously cannot do that with a fixed DRT that has a strong path-to-white transformation.

Cheers,

Thomas

Hi, I just wanted to chime in here to highlight something that may be apparent in some people’s minds but hasn’t explicitly been mentioned.

Path-to-white is not a goal. Path-to-white is a by-product of taking values which are not representable in the target gamut, and making trade-offs which overall achieve the ‘least visual difference’ from the input (this allows for the fact that if the target gamut encompasses the source, no transformation is required). I think we can all agree that having the correct sensation of brightness in the resulting image is more important than having the correct amount of saturation, said in another way, differences in brightness cause a large perceptual ‘error’ in comparison to differences in saturation.

I have built a system which allows you to tune the weight of saturation error and brightness error, and have validated the idea that faithfully representing brightness is far more important than faithfully representing saturation (though what I’ve landed on is not weighted purely by one or the other). Though the relative weights of these error metrics is subjective, I’m yet to come across someone who thinks an image which retains saturation at the expense of brightness looks more similar to the source than one which does the opposite.

If we had a target gamut which didn’t have a more luminous white than the primaries, there would be little benefit in desaturating in a ‘path-to-white’ manner. The fact that we are displaying this content with devices which have a specific gamut shape (namely that achromatic light can be represented with more luminosity than coloured light) is key in deciding what the transformation does. I think this is the reason why Troy is hammering home the point that you must keep in mind the destination gamut when crafting this transformation.

5 Likes

Hello Christopher,

thanks for this wonderful explanation. This is great to have you here ! Welcome to ACESCentral !

I have followed your work last year, especially the spectral rendering implementation into Blender.

Any chance to access this system ? Or see your results ? I am curious.

Best wishes for 2022,

Chris

2 Likes

Yes, I think this was one of the main points I was (trying to at least) make in my original post in this thread, and in some of the subsequent ones :sweat_smile:

This is an interesting fact that I hadn’t directly thought about before but after mulling it over I think I mostly agree that the fact that most gamuts we care about have brightest values along achromatic axis taken together with your note about caring more about brightness provides a high motivation for path to white… however, there are some other benefits about path to white in that even in a theoretical gamut with higher brightness in primaries, there would still come a point at which that extra brightness would run out and you still need to be able to represent a higher range… pushing everything towards the brightest primary doesn’t make sense as it completely changes the color intent and balance in a display-dependent way. Note that this still fits with the motivation for path to white not as a good thing itself but as a means to end like you mentioned above.

In any case, thanks for your post, I think it was well-said!

Thank you for the welcome.

I will see if I can share an image of mine passed through it with a sweep from high brightness error weighting to high saturation error weighting. It was designed for a different purpose than HDR>SDR mapping so isn’t directly relevant in this context, but there might be some insights to glean from it.

1 Like

I was mulling over this last night. In a space which looks like for example sRGB, but white is only 0.7 times what it is normally, the green primary is brighter than the brightest achromatic stimulus, which would certainly lead to some confusing results. I think the key thing here is that I personally consider differences in hue as ‘not an option’ so any input value would be constricted to the plane of equal hue (changing brightness and saturation). In this situation, if there was a very bright, moderately saturated green, I think it would make sense to increase saturation in order to represent higher brightness.

Going back to traditional media, this would be like giving a painter a black canvas, lots of bright, saturated paints, and a grey, but no white. It would be interesting to see what sorts of artworks come out of that - I have a feeling many might use the saturation (which carries with it the ability to express brightness) moreso than if they had access to white paint.

I also agree that even when achromatic isn’t brighter than any particular primary, there may be benefit in desaturating, but I think at that point we’d be leaning heavily on learned aspects of image production. If a bright red light goes dark grey in the middle, as compared to being a flat red, one thing we gain is greater ability to express tonality (variation in brightness, in this case specifically in regions outside of the target gamut) since we have a new axis to vary on (red to grey), but we have to rely on the viewer to understand that the dark grey means ‘brighter than the red’.

Fortunately with additive colour we’ll never need to account for such a space.

As much as display-dependent is a naughty word here, there is a certain aspect of this that necessarily has to be display (read: output gamut) dependent. Take the overly-aggressive desaturation of yellow. This is caused by the blue primary adding very little to the luminance of the resulting signal - said differently, red + green is very nearly as bright as white in spaces where the blue primary is not very luminous. Yellow feels like it desaturates aggressively because for 100% of the saturation of the original colour, you only receive a tiny amount of additional luminance. Where does display-dependency come in here? If your display has an unusually luminous (too green, not saturated enough) blue primary, then the desaturation of yellow becomes less aggressive, and counteracting for it in a fixed way would cause the opposite problem.

To illustrate, here is a side view of a rec.709 cube where each primary is scaled by its luminance and rotated to have the achromatic axis point straight up. Here you can see just how close yellow is to white in terms of luminance. Just be aware I threw this together now so it almost certainly has some problems.

2 Likes

Welcome @christopher.cook!

Yes, the HVS is more sensitive to brightness changes compared to chroma changes, this is the basis of chroma sub-sampling, YUV, Y’CbCr, YCoCg & co.

This sentence is interesting because depending how you read it, it is incorrect: In theory, if you design a system where the stimuli are generated in an “ideal” perceptually uniform space, e.g. “JzAzBz”, “CAM16L-LCD” (putting quotes because they are not perfect), the difference of some delta units along the lightness axis should be perceptually equivalent to the difference of the same delta units along another axis. Actually not matter the vectors, provided they have the same length, an observer should not be able to perceive a pair as being more different than the other.

Goes back to what I just wrote, which metric is being used for your system?

Agreed! Unfortunately, in our case, the white luminance coming from the display will always be that of the luminance summation of the primaries, so unless we limit peak white artificially, the primaries will always have less luminance.

On a very much related topic, Samsung engineers are trying to leverage the Helmholtz–Kohlrausch effect to increase brightness of displays while reducing power consumption: A New Approach for Measuring Perceived Brightness in HDR Displays

Is there anything making it like that the VWG does not keep it in mind? I could be wrong but I certainly don’t have the feeling that the transforms produced so far or last year discussions have ignored the rendering medium. Irrespective of whether they do it successfully or not, all the current ACES ODTs acknowledge the destination gamut, they are purposely targeting a specific device. Sure gamut mapping is crude, i.e. clipping, and colours are skewing, but they have not been engineered without thinking about a target device.

Cheers,

Thomas

1 Like

Thanks for the welcome @Thomas_Mansencal!

This is a good point - in the technical sense, the ‘amount of difference’ can be equal regardless of the axis of the change depending on the space you’re in. My thought here is that I’m not sure whether this is exactly the right context in which to be making the decision regarding image formation. While designing the system I’m working on currently, my concept of ‘error’ was regarding how similar or different an image looks (is perceived, rather than how the screen emits light) from the source, which includes the context of surrounding areas of the image. In this context, I feel like the greater importance of brightness representation over saturation is even more prominent. At the extreme, greyscale images make sense and are easily interpreted by the brain, whereas an image which only varies in chromaticity takes a significant amount of work to read.

The purpose of the system I’m working on is only to move values from near the target gamut (L not greater than destination white, and chromaticity not far beyond the destination gamut footprint), so I found the specific metric I used didn’t make a considerable difference as long as it has reasonable hue uniformity while desaturating and a reasonable prediction of luminance. I’m currently calculating error metrics in OKLab. One aspect that I have noticed is that it treats saturated reds and blues unfavourably, likely due to the un-accounted-for HKE contribution to brightness.

No, there is nothing that indicates that this isn’t the case - I don’t follow ACES very closely so forgive the ignorance there. My intention with this statement was just to reiterate its importance when making decisions like this.

2 Likes