Validity of ACES criticisms?

Because 3x3 fits are generally pretty poor for more chromaticity pure values. I consider the general 3x3 fits a continuum:

  • Reasonably acceptable - at the achromatic axis origin
  • Moderately acceptable - as values move away from the origin, but close
  • Poor - as we move out toward the edges of the locus
  • Nonsense - values fit to beyond the locus

This is also part of the domain confusion here, as a three primary based fit reprojects the locus for the camera “observer”, as below.

And this list is merely the isoluminant chromaticity aspect. We could probably argue that in terms of flicker photometric luminance the values could be considered along a similar scale.

Yes. Because a domain is different from a range. At risk of using a goofy analogy… a domain is a banana, while a range is a count of bananas. We can discuss slicing up the domain of a banana, but speaking of slices outside the banana is nonsense.

Subtle language nuance can lead to incorrect inference, so I try to use language that insulates against these inferences. A closed domain is a bounded limit, with nothing beyond. For example, if we forget that a 3x3 CAT is domain bound, we might consider plotting the results in accordance with the Standard Observer Illuminant E model. If we realize that the Standard Observer is domain bound, we’d also realize that in order to properly project it into a normalized CIE xy projection, we’d have to reproject the locus itself, to account for the entire reprojection of the domain.

In the end, tristimulus is tristimulus in terms of a numerical metric. The domains, on the other hand, matter significantly.

You are quite correct that using “from” and “to” is probably weak, given that I place value on the formed image, and in that light, I do not believe the image exists in the open domain tristimulus, but rather is formed via mechanics.

As for “range”, see above. I believe it is an inappropriate use that can lead to incorrect inferences.

Yes, more or less. A “creative flourish” is something that could be considered strictly “creative”, hence, according to Merriam-Webster “a decorative or finishing detail”. I’d be less inclined to place technical facets under the umbrella term, though.

Is this not covered by the two booleans of “Expand Chroma” and “Expand Brightness”? Are there other perceptual qualia here being overlooked?

Merriam-Webster:

any of the definable aspects that make up a subject (as of contemplation) or an object (as of consideration)

I’m not fond of “Display Rendering Transform”, hence I don’t use it, as it comes with some directionality, and lessens the influence of the image formation chain. Dye density layer crosstalk in subtractive mediums, or even per channel crosstalk in additive systems, is a critical and fundamental component in terms of the formed image / picture. That mechanic, as well as the result, does not rest in the open domain tristimulus, or in the case of actual subtractive creative film, in the spectral energy outside of the camera.

Saying “rendering” therefore, is both too general, and lessens the mechanics of the actual image formation in terms of mechanics.

I use the term “chroma” in the CIE sense, as I believe that’s the appropriate term for distance to an achromatic adapted axis.

colourfulness of an area judged as a proportion of the brightness of a similarly illuminated area that appears grey, white or highly transmitting.

Many folks erroneously use the term “saturation” here.

Laden is simply “heavy”.

Qualia is a term used in quite a few colour science papers, and has origins that track back a long ways to possibly 1675.

a property (such as redness) considered apart from things having the property
a property as it is experienced as distinct from any source it might have in a physical object.

Generally speaking, it’s a phenomenological term.

Likely redundant with the use of “appearance” and “percept”, but to clearly distinguish as the phenomenological aspect of human sense, not the erroneous idea of an attribute of an object.

percept: an impression of an object obtained by use of the senses

As above.

If the takeaway was simply “more control of the rendering”, then you gravely misinterpreted what was said.

I spoke of breaking down the image formation chain such that components can be isolated, as well as drawing a connection to the ICC protocol, where some of these facets are enumerated.

“Intent” from the ICC idea of “rendering intent”.
“Facets” being components.
“Flourishes” being largely creative embellishments.
“Qualia” being things that are specifically related to sensation in the phenomenological sense.

Hope this helps.

3 Likes

There has been some recent conversation on slack and twitter of “what is an image” and where / at which stage it exists in a “scene-referred” color management workflow. I am not sure if you read them or not but would love to know your take on it.

For instance, I now like to think that the file that is written / spit out by our render farm at work is not an image. It only becomes one once it gets through the image formation chain.

I am curious if you had had conversations like that at Kodak ?

This boat image with the quote has been shared on AC several times during the OT VWG. I would be curious to know what this sentence means to you “The upper image represents scene colorimetry, the lower image represents rendered colorimetry” ?

I think it goes hand-in-hand with this quote (from Slack) “For proper colour appearance, the colorimetry of the displayed image must always be altered from that of the scene”, right ?

rendered” ? “must be altered” ? Why is this necessary and in which way should it be do you think ?

Thanks for the insight,
Chris

As far as I’m aware, this was never a topic of any debate at Kodak. We certainly used the term “scene-referred image” regularly. We even used the term “image” to reference very much “unformed” images such as a “latent image” on a piece of exposed but unprocessed film.

The only time I can recall someone saying “it’s not really an image at that point” was when making the point that digital sensors are effectively analog capture devices that capture photons, convert them to electrons, and transport the electrical charge to an A&D converter. However, I think this distinction was primarily to stress the physics involved in electronic sensors and the comment was not at all related to some philosophical point about image states or image processing. In fact, we would often talk about the three stages of an imaging system as “image capture”, “image processing”, and “image reproduction.”

To be frank, this feels like an “If a tree falls in the woods, does it make a sound?” debate.

Firstly, ignore the obvious issue that the images are side-by-side and not stacked vertically in the posted picture.

The primary point is that exactly reproducing the original scene’s XYZ values on a reproduction medium is generally not going to produce a pleasing image. Specifically, if you measure an object’s XYZ values in a scene, then reproduce those XYZ values exactly on something like a monitor, the scene and the reproduction will not appear the same. This is due to differences in the viewing environment and its effect on a viewer’s state of adaption when viewing the objects.

In the case of the scene, the viewer is typically adapted to much higher levels of general luminance, surrounds aren’t discernably different than the image area. In the case of the reproduction, the general luminance is typically much lower, the image has a surround that’s distinctly different than the image, and the imaging system introduces flare. Not compensating for these differences in environment and the effects it has on a viewer’s state of adaptation will cause a viewer to perceive the same colorimetry very differently in each environment.

The compensation for these viewing environment differences is generally described as “color rendering.” Color rendering can also include preferential, or creative, adjustments but it’s primarily altering the scene colorimetry in order to compensate for differences in the viewing environments of the original scene and the reproduction. I highly recommend “Digital Color Management - Encoding Solutions” by Giorgianni and Madden for a more in-depth description.

As a bit of an aside, you’ll often hear people say an image isn’t “scene-referred” because the image’s colorimetry doesn’t exactly match the colorimetry as measured in the original scene. In fact, the image’s colorimetry isn’t what dictates its image state, but rather the viewing environment associated with that colorimetry is what defines its image state. In the extreme case, one may have two images, both with the exact same code values, representing the exact same colorimetry. If the stated viewing environment for each image is different (one consistent with a scene’s environment, the other consistent with a reproduction environment), then the image states are different.

Long story short, the viewing environment and the state of adaptation of a viewer of an image provide critical context for how an image will appear to that viewer. If the goal is to roughly make an image reproduction that appears the same as the original scene then compensations for the viewing environment differences must be made. This is largely the role of color rendering.

3 Likes

Can we extend this further?

If we non uniformly distort the fit chromaticity values from a sensor’s quantal catches, the resulting image will be “less than optimal” for many reasons, including but not limited to:

  • The range of the Standard Observer IE fit tristimulus chromaticity values cannot often be expressed in the medium.
  • The range of the Standard Observer IE fit tristimulus chromaticity values often maintain no relationship to actualized Standard Observer pseudo-neurophysical responses as the fit chromaticity values are nonsensical.
  • The ratios of the sensor values are often limited in a technical sense, resulting in undesirable distortions of chromaticity angle, chromaticity purity, and luminance. EG: Channel clipping at the low or higher end yield distortions in measured colourimetry.
  • The complex relationship of image / picture reproduction is different to “measured” values in the aesthetic preferential sense. Example includes chromaticity angular distortions on skin, as well as chromaticity purity distortions via crosstalk. 123
  • The “optimal” reproduction of tone.6

Note, the above isn’t related to the more direct technical attributes of appearance constancy of a formed picture, such as some form of measurements of the Mona Lisa. These sorts of technical manipulations are of course critically important, but I believe it lacks the nuance of the subject at hand.

The addressing of researched discrepancies of “as measured” metrics versus “as produced and manifested in a picture” metrics.

Given that important facets that must be present in the formed image / picture are not present in values from a colourimetric fit of sensor quantal catches, it would seem that the debate is higher up the chain?

Are we sure we are speaking of a singular tree?

For example, part of the critical aspect of image formation in creative film is the density relationship of the dye layers; as the dye level decreases, the dye to achromatic bulb projection ratio shifts. This is no longer a mere matter of intensity of a given chromaticity, but rather:

  • Chromaticity attenuation as the dye densities deplete.
  • Chromaticity angular shifts as the dye ratios interact via crosstalk.
  • Luminance shifts as the dye ratios interact and shift, and transition to zero density.

It would seem that the fundamental interactions in creative film reproductions, and arguably the entire legacy of painting, are so critically different from mere colourimetry of measured spatial representations as to be their own unique class of “thing”.

We could indeed toss all of the complexities and nuances that were studied for over a hundred years in terms of picture creation etc. under the banner of “rendering”, but it would seem to be a grave injustice to suggest that we are merely “rendering” the colourimetry. Are we not indeed reshaping and changing the colourimetry in monumental ways? The importance of chromaticity attenuation on skin tone reproduction alone should be enough to give us pause?

And if we are not merely “rendering” the colourimetry as derived from some fitting of quantal catches, what then are we performing?

Could this be deeper than mere state of adaptation, and into visual segmentation?45

This passage seems to be doing a lot of work? Are we certain that a picture is indeed the mere simulacrum of “standing there”, or is that a potentially seductive myth given the above discrepancies that have been researched rather extensively?

The term used here is “image”. Are we talking about the formulation of an image / picture or “standing there” measurements? Are they the same thing?2

References
1 Zeng, Huanzhao. "Preferred skin colour reproduction." (2011)
2 Hunt, R. W. G., I. T. Pitt, and L. M. Winter. "The preferred reproduction of blue sky, green grass and Caucasian skin in colour photography." *The Journal of Photographic Science* 22.3 (1974): 144-150
3 MacAdam, David L. "Quality of color reproduction." *Proceedings of the IRE* 39.5 (1951): 468-485
4 Ekroll, Vebjørn, et al. "The natural center of chromaticity space is not always achromatic: a new look at color induction." *Proceedings of the National Academy of Sciences* 99.20 (2002): 13352-13356.
5 Anderson, Barton L., and Byung-Geun Khang. "The role of scission in the perception of color and opacity." *Journal of Vision* 10.5 (2010): 26-26
6 Jones, Loyd A. "Photographic reproduction of tone." *JOSA* 5.3 (1921): 232-258
5 Likes

This came just right, Steve Sasson, Electronic Engineer at Kodak and inventor of the Digital Camera talking about his invention: How Steve Sasson Invented the Digital Camera | PetaPixel

I can only appreciate the simplicity of his wording that anyone can understand and does not involve translating ill-defined bespoke terminology.

As I asked on Twitter, what is the benefit of adopting this mental model? Does it make you more performant, more cost effective?

I could trivially point out the benefits of adopting a scene-referred workflow at many points in the CG authoring chain. The terminology is defined in ISO standards: https://www.iso.org/obp/ui/fr/#iso:std:iso:12640:-5:ed-1:v1:en:fig:1

What are the benefits of suggested mental model over it? How do you call it?

My tuppence. If you look at this “image” mental model under the scope of short-term economic benefits, you are looking at it with the wrong lens.

Thinking of an image-referred workflow opens possibly the door to a new protocol where faithfulness of creative intent, inversibility and operations before and after the image is formed are improved/allowed.

What I love about this mental model is that two “stupid” questions such as “what is an image ?” and “where does the image exist in a scene/display workflow ?” shake up my own beliefs and force me to think out-of-the-box.

Most of the supervisors I have worked with do not know what happens between the DCC software and the Render View and what happens when we tweak RGB sliders. So even if this conversation leads nowhere, I can only appreciate how it has forced me to think about things differently and to understand better my job as a lighting artist.

Again if the scene/display workflow fits all your creative needs, no worries, I get it. But I honestly do not understand the pushback of thinking about different protocols like we did a more than a year ago on the Miro board.

Regards,
Chris

By naming it the “image” mental model it almost looks like the scene-referred workflow does not deal with images, which is obviously incorrect, quite the opposite in fact: There are many images in the processing chain and that they are in different states. This bypass entirely the disagreements and pedantry related to the definition of an image, which depending on the context is different.

We are working in a very competitive industry and when I look at something new, it is often to understand if the people I work with and I can benefit of it. I’m glad to hear that this is useful for you.

The push-back against the suggested model is a mere proportional, actually polite one comparatively, reaction to the pushback the scene-referred workflow receives.

Cheers,

Thomas