And per channel doesn’t do anything even remotely close to this.
It’s not; it’s just a skewed and posterized variant, that destroyed the intention of the ratios.
And per channel doesn’t do anything even remotely close to this.
It’s not; it’s just a skewed and posterized variant, that destroyed the intention of the ratios.
Per-channel tone mapping is exactly the mapping which affects mapping of whole scene luminance too. How that mapping should be done? It is a completely subjective/creative. Even compensation of viewing environment is a subjective transformation too. And doesn’t exist precision ways for doing it. What is even remotely close for “correct” mapping?
Because the per channel approach on anything beyond an inverse EOTF changes the ratios of light mixtures in the source encoding.
When the ratios of light mixtures change across the surface of intensity, it changes in a nonlinear and non uniform manner.
You used the word luminance here. The light mixture in the working space has a specific chromaticity, and therefore luminance, based on the ratios of the three lights. When the lights are adjusted on a completely independent basis, the intention of the light mixture in terms of chromaticity is completely lost. Again, to be clear, per channel approaches will, throughout an image emission range:
Then why bother doing colour formation matrices at the camera? Further, imagine picking a colour in a DCC and having it end up a completely different light mixture colour.
Is it all creative?
We can adjust and compensate for certain facets of perceptual-like responses. This is 100% correct.
However, at the point we throw technical and analytical changes into the same bin as “random subjective / creative”, we might as well give up.
Why not just generate random light mixtures? Why not have blue lights turn green?
Either we do colour science things to get the camera to try and represent what it captured in terms of colourimetry, or we don’t. Either we honour the intention of the light mixtures in the working space, or we don’t.
Why not make all skin tonality turn yellow? Magenta? Cyan?
The colorimetry of original scene have to be changed to fit target media dynamic range. What changes of “Hue” and “Saturation” expected by the viewer? Should both chromaticity and saturation be always kept? There are some complex color appearance and color prediction models. They are complex and I didn’t remember result which was aesthetically pleasant. May be you have a good examples? On the other hand we have some history of film photography/movie/paintings… What if our eye want (or get used to) pleasant color reproduction more than colorimetrically correct (if it is even exists) one?
“Hue” and “Saturation” in terms of chromaticity and chromaticity-linear paths are changed. I can be wrong but I expect some non-linear continuous nature of changes of color ratios. But why do you call that changes “broken” and shifted in “arbitrary” ways? We don’t talk about random changes. Color mixtures are shifted in the directions and with power defined by non linear mapping function and primaries of the working color space. May be the problem lies in RGB working color space primaries?
By the way, often camera matrices are built (optimized) on some sub-set of subject-important colors (ColorChecker24 etc)? But we talk about scene reproduction colorimetry. Capture and display devices should be colorimetrically correct devices as much as it possible.
If an image maker picks a colour in a DCC, 99/100, do they expect the mixture to randomly change?
Less of a discussion. Perceptual mappings are an additional consideration here. Remember, per channel isn’t some sort of engineered output! It is random based on the shape of the curve, and is applied irregularly based on the ratios of the mixture, and the chosen working space primaries!
Per channel does not do this. Again, it is:
It’s accidents all the way down, as opposed to engineered solutions.
So you expect a blue to turn magenta or cyan, a red to turn yellow or magenta, and a green to turn yellow or cyan, per curve, per working space primaries set?
Look at the variables above and try to arrive at it being anything other than an accident. Utterly arbitrary, and distinctively digital RGB artifacting.
Try it! Plot the results in a chromaticity space such as 1976. Most folks who do so are completely surprised that the assumptions do not hold up in any way.
Why bother when the actual application of the rendering is completely arbitrary and not delivering the mixtures represented in the working space.
There are some hard limitations here when approaching it from an engineering standpoint at a given bit depth, but random and arbitrary skewed output isn’t an entry point to these sorts of discussions.
Thanks guys. Lots of interesting thoughts here ! I honestly believe we’re making some progress. I’ll try to reply to each point to the best of my abilities. So please bear with me !
@alexfry yes, we have thought of the LMT implementation. Thanks for pointing that out ! Issue is that we’d need someone with the proper knowledge to do so (aka a color scientist/TD). And I think lots of small studios/freelancers would be in the same situation, right ? My take is this : would it be fair to say that a Color Management System (CMS) should be able to display properly any values it gives access to, out-of the-box ? I think that’d be a nice improvement. Of course, devil is in details and what do I mean by “display properly” is an open question.
@Alexander_Forsythe thanks for the answer ! This is great ! Let’s for the sake of argument that we agree that mapping from scene to display is a creative choice. What is the best starting point here ? Out-of-the box, no LMT, which option seems more reasonable to you ?
Please remember the scenario where we have hundreds of CG artists from layout to compositing putting colors in their scenes. As a supervisor, which starting point would you rather have for your teams ? Could there be a bucket #3 where we compensate scene values because the display rendering transform is twisting hues ?
@garrett.strudler , cool to have you here ! Don’t apologize. I am the king for making false assumptions ! Yes, I’d have expected my blue neon to go to white earlier and I agree this is debatable. But I disagree on the second argument : about the re-light. On any CG movie, this argument would be true. But not on the renders I provided. For two reasons :
Again, as an artist, what would be the ideal starting point here :
Look at the cheek, the neck and the t-shirt :
We are all on the same mission here and let me give you a hint of what M. Giorgianni has written on the topic :
Appropriate gamut mapping would then be applied to transform those colors as needed for real outputs. […] It would be expected, then, that some form of gamut mapping will be required in going from the Output CES to any real output device or medium.
I think the key is not to think only about luminance range but also about the gamut volume. We go from one volume to another. Again by M. Giorgianni :
It allows information to be encoded in a way that places no limits on luminance dynamic range or color gamut. Any color that can be seen by a human observer can be represented.
Hello @ruslan.sorokin and welcome to acescentral ! Again, for the sake of argument, let’s say we agree that the mapping is subjective/creative. Did you have a chance to look at this thread where I tried to point at some of the issues we are talking about ? Please have a look at Nick’s gifs and Jed’s videos here. It is important that we agree that per-channel’s path to white is an accident, maybe even an happy one.
But as an artist, I want control. If I like skews, for an explosion maybe, I 'd rather introduce them as a creative grade or a LMT. Again, have a look the renders and comparisons here : what is the most reasonable starting point ? If I want a blue sphere to look purple under the sunlight, I think it should be a decision/choice, not a default behaviour.
I have also done my homework and asked a dozen of supervisors about a radical change in the Ouput Transforms for ACES 2.0. Mostly from worldwide VFX and Animation studios. I have been given pretty much every time the same answer :
For three main reasons :
As a studio, we are definitely interested by Jed’s experiment and any quality improvement that could be made to the Output Transforms.
Happy to discuss,
Camera, display and software were engineered. We use color science to solve engineering tasks to satisfy our aesthetic requirements. “Chemical engineers” done the same thing when invented emulsions. So, term “engineered” is a little bit confusing me.
If we talk about changing image contrast. This is not colorimetrical term, right? There is no “natural” transformation. There are only engineering ones. Per-channel is the only simplest one. No more engineered than another.
Film emulsion has 3(4) layers. Each layer has own spectral sensitivity and can be characterized by some transfer function. So, the behavior is pretty much like “per-channel”, no?
I have thought of a couple more things overnight.
I always go back to this simple example. What’s happening here ? By increasing the exposure on a sRGB blue primary, the blue channel is arriving first at 1 (or 100% emission) and cannot go beyond. Indeed displays have limitations and do not magically go to white. Then the red channel and green channels catch up creating a hue shift towards purple.
Shall we agree on the following :
I think it would be a step forward if we agreed on these three statements.
I have said earlier in this post that adding 7-8% of green would be a compensation for this behavior. I think it is worth nothing that by doing so, we are just changing the skew direction and not necessarily fixing the issue.
The epiphany I have had so far with this VWG is that the path to white must be engineered on a digital display (it was also engineered for film, agreed, but I would not remotely compare per-channel with film emulsion). We always come back to this fundamental question :
What should happen to values that are even more intense than 100% emission ? Just leave them stuck at the maximal value ?
And since we are as well having this great debate about film vs eye, I thought of another silly experiment :
Finally, per-channel is indeed engineered (nice catch @ruslan.sorokin ) but shall we agree that it is the responsibility of this VWG to at least investigate and discuss more elaborate transforms ? Since respecting the creative intent is our goal here ?
A side note : I would not literally spend hours writing these posts and providing these examples if I didn’t care about ACES. I think this VWG has some of the best people in the industry which is why I have high hopes/expectations for the outcome.
Update : I forgot to share with you a beautiful answer from one the lookdev supervisors I contacted.
[…] the calibration of an asset should be completely independent of the quality of the Output transform, of the tone mapping and in the broad sense of the look (CDL, LUT, etc …) applied to the images.
Jed Smith has done another brilliant video that I wanted to share with you. If every single person from the VWG could have a look at it, I think it’d be a good thing. It is only 40 frames long and it shows us the path to white on ACEScg samples through the current Rec.709 ACES Output Transform. And it is awesome.
During the TAC meeting, it has been said :
There’s two implementations here […] Whether you should facilitate the maximally saturated output of a given color, whether you should enforce a desaturation towards white in order that you get these kind of nice hue property […]. And those two things directly are competing because one desaturates, the other one requests you not to […].
Although I do not disagree with this statement, I think it can be misleading. I think there is only one acceptable solution : not collapsing values at display on their path to white. I would rephrase it this way :
I have made some screenshots of the video to explain as best as I could.
Frame 1 : we start with low exposure values. So far, so good.
Frame 14 : red values start to collapse on the display limitations. Because a Rec.709 display monitor has limitations.
Frame 19 : blue and purple values collapse on the display limitation. Pay attention to the hue deviations/twists.
Frame 21 : green values collapse on the display limitations.
Frame 26 : The per-channel “accident” kicks in. Values “bounce back” from display limitations in order to go to white.
Frame 32 : Remember the original samples : they were a sweep of the whole spectrum. On their path to white, they collapsed into 6 values (aka the notorious 6).
You have the right to like this behavior. But I really don’t think this should be the default output transform. I think we can and should do better. We are trying to reproduce faithfully a scene by taking in account the display limitations, right ? And in order to do so, we need to compress values, which per-channel doesn’t do (well enough/at all).
And how we compress these values nicely/smartly/elegantly is actually a point of debate. Exactly like Lars Borg said in meeting#5 :
The effectiveness of this method will mostly rely on the quality of the gamut compression.
This is why the subjective/creative argument about per-channel does not work for me. I really don’t think that collapsing values is about taste. And this is what Jed is trying to address with his naive DRT.
I am more than happy to hear suggestions/ideas/comments on how things could be handled differently. But I don’t thing we can demonstrate things more clearer than that.
Finally, I would like to quote Rod Bogart from the TAC meeting :
the efforts that we’ve been going though, over the years, have led to this, that we couldn’t do this first. […] And now that we’re here, it’s time.
Now is the time do something awesome. Here is the Nuke script if you want to play with it.
ACES_Rec.709_Swatches_Exposure_Visualization.nk (146.9 KB)
Update : Jed has done the same video with his DRT so we can compare. Thanks Jed !
And for completeness, two final images (thanks again Jed !).
A pure AP1 hue sweep rendered through the ACES Rec.709 output transform :
This image is the same but the source with 90% of the AP1 gamut volume as input.
I’m having a lot of fun making plots and visualizations lately. Here is another one that maybe shows more clearly what is happening.
What is happening there?
Just thought I’d share this too in case it helps anyone visualize what is happening more effectively.
As usual, here’s the Nuke setup that generated that plot if anyone wants to play with it.
plot_aces_rec709_output_transform_rgb_six_rotate.nk (133.3 KB)
And here is a few more stills that show what is being clipped beyond the display gamut cube.
At 0.9 times the AP1 gamut boundary:
And at 1.0 times the AP1 gamut boundary:
facilitate the maximally saturated output vs enforce a desaturation towards white
To better understand the behavior of each approach, and therefore the pros and cons, I put together … you guessed it: some more plots!
Here’s the setup. We have a 64 frame animation, featuring two exposure sweeps from 0 to +8 stops, of a 6 patch hue swatch of primary and secondary colors, and a second exposure sweep of the same image with a 15 degree hue rotation. For clarity, each hue swatch is a linear ramp from 0 to full color in x and a linear ramp from full saturation to 80% saturation in y.
There are 4 videos with different display rendering transforms.
As usual, here’s the nuke script that generated these plots. overexposure_comparisons.nk (24.4 KB)
A couple of observations.
One of the biggest arguments for me in favor of a chromaticity preserving display rendering is more consistent rendering of color in highlights between SDR and HDR. I’m guessing here without an HDR display, but I think the hue twists inherent in a per-channel display rendering approach would be very difficult to undo when using an HDR rendering.
The only person I’ve seen mention this is @daniele, on the colour-science slack, so I thought I’d bring it up here.
4:18 PM Saturday Jan 30th
Daniele Siragusano I am not saying HDR and SDR should be identical. But a yellow candle in SDR should not be orange in HDR I guess
At least I know many colourists which are confused when that happens.
Here are some demos of “maximally saturated” with respect to the display gamut volume, without any chromaticity compression of any sort. The values are engineered to remain “fixed” at the limits of the display gamut boundary.
The following images use a generic S shaped aesthetic tonality curve. It should be noted that when using a chromaticity preserving rendering approach, the shape and form of the curve is of much lower consequence than than range it attempts to compress to the display.
Note that this is for over exposure, and the same problems arise for under exposure.
The EV presented here are relative to the source image’s default encoding.
+/- 0 EV:
Here is a representation of the values that are out of gamut at a mere +1 EV:
Great videos, they showcase well the orthogonality between no/path to white.
It is not necessarily HDR related, people have for example complained about how difficult it is to get saturated yellows with ACES SDR transforms for years. I would rephrase that more as some people wants/need to get very close or even touch the volume corner. With a transform that aggressively desaturates everything in the highlights, it is super hard, it is already with ACES OTs.
The core question to me is how we enable both?
One thing worth trying is a no-path to white chromaticity preserving transform with a path to white LMT that is effectively a hue-keyer allowing you to selectively control which hues are allowed to hit the full extent of the volume and which ones are not. I have done that at times
Conceptually an LMT is a transform which should be the same for all viewing condition (a scene-referred edit). The path to white might be different for different viewing conditions. It might be difficult to get a balance here if you do a binary split.
Yes the current LMTs are scene-referred but I’m also thinking about output-referred ones here, before the final display encoding. So you end-up with a single RT and a stack of output-referred LMTs that can be used to tweak the appearance on different displays instead of forcing that from a under the stack which ultimately is impossible to do with full control.
Without clear design requirements, we won’t get nowhere and we’'ll end up chasing our tails/running in circles.
Then we’ll talk solutions.
It strikes me that gamut compression concerns are adjacent to perceptual concerns. Do you see a way to pull out perceptual facets based on the internal “intentions” of the models? EG: Can a perceptual facet be negotiated at the very last stage in reconciliation with the display?
Something that was not noted in the issues on Dropbox Paper is the hard time to get saturated yellows with the current ACES DRT which was reported quite often.
Here is a video showing the current volume filled as exposure increases (sorry for the low sampling quality, I don’t have Jed’s budget :D)
Note that the distortions are caused mostly by the 3D LUT and interpolation.
And for good measure, Jed’s Naive DRT with default settings:
Note that the bias can be adjusted to get closer to the boundaries if required.
It is of course better viewed directly in Nuke: DRT Cubes · GitHub
It becomes tricky from an architectural and user experience standpoint if you start to introduce a stack of transforms, which together form the output transform.
I think what we are doing (scattered around this forum) is collecting parameter for a possible set of viewing transform. I think the real work is to agree on a balance of those parameters for a default viewing transform, knowing that many parameters (or even all) can only be set with a large compromise. A small group could agree on such tuning of those parameters, but as soon as you widen the discussion group it becomes unmanageable. Hence the ability to swap output transforms could save us here.
We should never forget that the net experience of a pipeline is a complex interplay of working space definition and output transform.
I personally do not put so much weight into the display-referred state, because we are here in the domain of image creation. So a skilled person can tweak things. Some edge cases can be loaded onto the shoulders of the content creators, as long as the pipeline is elastic. This can lead into a much simpler design.
…And I know that you would disagree :-).