<Range> Node

[jim_houston]

Since Doug is not presenting both points of view, I am sorry I think I have to responsd in line.

On a ‘lot of history’ – this conversation was never concluded and we ran out of time (which is why the CLF 2.0 spec was never finished and is listed as DRAFT)

Yes, matrix allows scale and offset, but it doesn’t allow extraction of a portion of a float range
and only work on that range. (the RANGE node is for that purpose and it still works. It is the implicit clamping at one end that is the problem – the workaround is to always use all 4 metadata items but the spec should not allow an implicit operation that causes the math to go off.

Again my objection was to the implicit one-side clamping, not about the presence of clamping.

on point #6 and this previous sentence, another solution is to make explicit the clamping behavior
ClampHi, ClampLo when needed. Adding another node with a different behavior is also an understandable approach but it needs to get back to the original intent on handling the full float range (not assuming float=1.0 scale) I propose deleting the objectionable text in this thread. The focus should be on that issue not the general existing of clamping – and no it is not a naming disagreement, it is about the fundamental math.

There should be a note about use of the matrix 3x4 node to scale and offset floats in an unclamped fashion. I agree that Clamping is essential.

I am sorry to say I may not be available until late in the call and request that this discussion
be deferred until I can join.

Thanks,

Jim

@doug_walker

There is a lot of history on this topic and many hours were spent by the committee during the writing of the v2 spec back in 2014.

Here is my attempt at a brief summary of the relevant points:

1. Fundamentally, the spec provides two ways of applying a shift and scale operation: Matrix and Range. The difference is that the Matrix does not clamp and the Range does.
2. When the v2 spec was written, Jim objected to having the Range always clamp and so we added the noClamp style option. However, this is purely a convenience for CLF authors (for example, under the hood OCIO will convert a noClamp Range to a Matrix, functionally they are no different). Also, there need not be any performance hit to using a Matrix since implementors can easily optimize the case where the off-diagonal terms are 0 (as does OCIO).
3. As much as we all strive to avoid improper clamping, clamping is needed in order to accurately implement certain transforms (including some ACES transforms). So CLF absolutely needs a way to do this.
4. There was an assertion during the call that Range cannot map [-0.5, 222] into [0,1]. However, that is incorrect, it can do that and furthermore allows the option to clamp or not.
5. The objections that have been raised seem to be not fundamental issues with the math but more with the “aesthetics” of the design. Please keep in mind that this was a committee effort going back over a decade and that it won’t be possible to please everyone on naming. If we were starting from scratch, I would absolutely support the idea of further discussion about naming and perhaps doing a poll with various options. However, we’re talking about v3 of the spec, and a number of companies have implemented it at this point (it’s also in OCIO now). At the end of the day we want more people to implement this and I don’t think it helps the cause to signal that the syntax is open to being redone at every revision of the spec.
6. There was some discussion during the call about the formulas for the case where only the min bounds or only the max bounds were specified. We added those formulae in the v2 spec to clarify for implementors what to do if they get a file that does that. However, there is only one important case of that type, which is when one needs to clamp at the low end or high end but not both. If people want to remove those formulae, we could add a limitation which is to say that if a Range only provides one set of bounds (i.e., either min or max), that it cannot shift or scale.

with respect,

Doug

[sdyer (Scott Dyer)]

@jim_houston

the workaround is to always use all 4 metadata items but the spec should not allow an implicit operation that causes the math to go off.

Again my objection was to the implicit one-side clamping, not about the presence of clamping.

What if we enforced that when style is ‘noClamp’ (even when only min or max pairs are provided) then no clamp is applied. Would that work? Then nothing is implicit because you are explicitly saying not to clamp.

If we went further and listened to @Greg_Cotten, we could require that style be set, which would require explicitly stating the intended behavior.

Side note: What might be a practical use case for specifying just min or max pairs and not wanting it to clamp?

[sdyer (Scott Dyer)]

@jim_houston

Scott: please review the spreadsheets I did at that time. They were about the calculations
of floats going wrong under the implicit missing item equations.

Yes I did review this but I had difficulty making sense of it for the min/max only cases when mixed with in/out bit depth differences. Perhaps if I had a practical example to help wrap my head around what the intended behavior is rather than just looking at numbers - then I might be able to better grasp the numbers to see some meaning in the current behavior.

For those interested in taking a look, here is the spreadsheet Jim is referencing: range_node_eval_v4.xlsx (53.6 KB)

[Thomas_Mansencal]

@sdyer

Then nothing is implicit because you are explicitly saying not to clamp.

It is certainly much preferable.

[doug_walker]

Hey Jim, it looks like the website messed up the formatting of your message. It’s not clear what you’re objecting to or what your proposal is. As Scott suggested, I think it would help to have a specific example to discuss.

[jim_houston]

My proposal was to delete the text in the Range node that deals with the assumed scaling behavior if one piece is missing.

An example is for an input of 0 to 10,000 nits and an output of 0 to 1023, the scale factor that gets calculated is (1023/10000) = 0.1023 when all four parameters are present. This is the correct conversion
from a 10000.00 float to 1023. With a 5000n input, RGBout = 5000 x 0.1023 + 0 - 0 x 0.1023 = 511.5 the right answer.

Exactly per the spec, if only the max values are specified, the scale parameter becomes ( 1023/ 1.0 ) or 1023 X as a multiplier. This is 10000 Times OFF. This is the damage from assuming all floats operate in the 0 to 1.0 range.

With an RGBin of 5000n and following the spec exactly with the equations as specified for the max only case…
RGBout = MIN(1023, 5000n x 1023 + 1023 - 10000 x 1023) or MIN ( 1023, -5113977 ) gives the wrong
number in the full float input situation. (The 511 is there but just of completely the wrong magnitude).

My proposal is to delete rangescale(float)=1.0 and everything after it and require all 4 parameters with a specific style addition of ClampHI or ClampLO. This minimizes the changes to the node input but if it still seen as necessary to keep the damaged math of the RANGE node as is, create a new RANGEF node with a note about the assumption in the existing RANGE node.

Jim

[doug_walker]

Thank you Jim. When I plug in your example values I get a different number. But in any case, I’m open to prohibiting the use of only one side (max or min) in a case that does not allow a simple clamp.

We’ll discuss at the meeting today …

Doug

[nick (Nick Shaw)]

@jim_houston

An example is for an input of 0 to 10,000 nits and an output of 0 to 1023, the scale factor that gets calculated is (1023/10000) = 0.1023 when all four parameters are present. This is the correct conversion
from a 10000.00 float to 1023. With a 5000n input, RGBout = 5000 x 0.1023 + 0 - 0 x 0.1023 = 511.5 the right answer.

Exactly per the spec, if only the max values are specified, the scale parameter becomes ( 1023/ 1.0 ) or 1023 X as a multiplier. This is 10000 Times OFF. This is the damage from assuming all floats operate in the 0 to 1.0 range.

It took me a little while, but I now follow your example, @jim_houston, and get the same result you do. I did not realise initially that your scale=1023 was coming not from maxOutValue=1023 but rather from outBitDepth=10 which is implicit, but not explicitly stated, in your example.

It does seem to me that the maths in the current spec is just simply wrong (or at least counterintuitive) for the single ended clamping cases. I have (I believe) a simpler, float only, example which illustrates this.

maxOutValue and maxInValue are not included at all in the calculation of scale. So if inBitDepth and outBitDepth are both 'float' then:

scale = rangescalar(float) / rangescalar(float) = 1.0

If I then set maxInValue=2.0 and maxOutValue=10.0, I might expect the operation to simply multiply input values by 5, and clamp at 10.0. Therefore 0.5 input would produce 2.5. However, testing this in Python, I get:

>>> maxInValue = 2.0
>>> maxOutValue = 10.0
>>> scale = 1.0
>>> RGBin = 0.5
>>> min(maxOutValue, RGBin * scale + maxOutValue - maxInValue * scale)
8.5

Is this really the intended behaviour?

It is effectively using implicit values of minInValue=0.0 and minOutValue=8.0 which is not what I think most people would expect. Intuitively you might expect 0.0 to be used for both.

Indeed in @jim_houston’s example for the result in the second case to match that in the first, implicit minimum values of zero would have to be used. But in fact an implicit minInValue=9999.0 is used.

[nick (Nick Shaw)]

On a separate subject, I am not particularly keen on this sentence in the spec:
“If the input and output bit depths are not the same, a conversion should take place using the range elements.”

Who “should”? Does it mean that people creating CLFs should add e.g. an explicit range mapping from 0-1 to 0-1023? Or does it mean that CLF implementers should apply bit-depth based range scaling automatically? It kind of means both, which is a bit confusing.

If a CLF creator includes a Range node with inBitDepth="32f" outBitDepth="10i" if they provide only, for example, minInValue=0.0 and minOutValue=0, a normalised float to 10-bit scaling will be automatically applied (with negatives clamped). If they provide all four range values, they need to be values which correctly apply that scaling, and inBitDepth and outBitDepth are ignored in the calculation of the mapping.

But maybe it’s only me who finds this confusing…

[doug_walker]

Hi Nick,

The only scaling in the formula for the “single-sided” case is what is necessary for the bit-depth scaling. I think there is a logic to that since you would need “both sides” in order to infer both a non-default scaling and an offset. So the “single-sided” case is basically a simple offset that is corrected for any bit-depth difference. So in your example, maxInValue does get mapped to maxOutValue, which I think is the primary expectation. Your example input value of 0.5, which is 1.5 less than maxInValue, gets mapped to 8.5, which is 1.5 less than maxOutValue. So I think it is a plausible result.

But as I wrote farther up in this thread, I agree that the “single-sided” case is probably not very useful for anything other than a simple clamp. In fact, in the interest of moving this forward and trying to bring us to consensus, I’ll go ahead and make a concrete proposal:

In Scott’s PDF on OverLeaf referenced earlier in the thread, change equation 3 from:

out = MAX(minOutValue, in × scale + minOutValue − minInValue × scale)

to simply:

out = MAX(minOutValue, in × scale)

and change equation 4 to:

out = MIN(maxOutValue, in × scale)

And add a note such as

Note: If only the minimum values are specified, the values must be set such that minOutValue = minInValue × scale. Likewise, if only the maximum values are specified, the values must be set such that maxOutValue = maxInValue × scale.

(One could argue that only the minIn or minOut is necessary, but I think it is important for readability to include both.)

I think that would address the issue that you and Jim raised. (Yes?) It is a change from the v2 spec but it’s minor enough that I’m not aware of any real-world use-cases that would actually break due to that change.

Regarding your other post about scaling, I agree that sentence is confusing. Actually, I don’t think that sentence is even necessary so perhaps the best thing is just to delete it.

Doug

[nick (Nick Shaw)]

@doug_walker

I think that would address the issue that you and Jim raised. (Yes?) It is a change from the v2 spec but it’s minor enough that I’m not aware of any real-world use-cases that would actually break due to that change.

I think it would address the issue, as long as it is clearly documented. Currently it is slightly unclear whether the document is user-facing or developer-facing. From a user perspective it would be easy to make assumptions about expected behaviour, such as implicit minimum values of zero when omitted. It would be better if the behaviour was clearly described in the text. At the moment, it is necessary to run various numbers through the provided equations (as @jim_houston and I did) to find out what the behaviour is.

@doug_walker

Your example input value of 0.5, which is 1.5 less than maxInValue, gets mapped to 8.5, which is 1.5 less than maxOutValue. So I think it is a plausible result.

Plausible, yes, once you analyse what the maths does. But I don’t think that a pure offset, based only on the difference between in and out values is what people would necessarily intuitively expect from an operator called “Range”.

out = min(in, maxInValue) * (maxOutValue / maxInValue)

is the maths I would intuitively expect, as only scaling seems more logical to me than only offsetting.

However there is no obvious equivalent for minimums only, particularly as those could often both be zero.

[Thomas_Mansencal]

@nick

But I don’t think that a pure offset, based only on the difference between in and out values is what people would necessarily intuitively expect from an operator called “Range”.

No for sure! To me and without looking at the doc I would expect it to be equivalent to those:

• MapRange: https://docs.unrealengine.com/en-US/BlueprintAPI/Math/Float/MapRangeClamped/index.html
• SetRange: https://download.autodesk.com/us/maya/2011help/Nodes/setRange.html
• FitRange: https://www.sidefx.com/docs/houdini/nodes/vop/fit.html
• LinearConversion: https://github.com/colour-science/colour/blob/a992ee574f969619554735ff60487d78838b9cd4/colour/utilities/array.py#L848

Cheers,

Thomas

[nick (Nick Shaw)]

@Thomas_Mansencal

No for sure! To me and without looking at the doc I would expect it to be equivalent to those:

To be clear, it is equivalent to those when all four values are given. It is the behaviour when only two are given which is perhaps not intuitive.

[Thomas_Mansencal]

@nick

To be clear, it is equivalent to those when all four values are given. It is the behaviour when only two are given which is perhaps not intuitive.

Argggg, I have been beaten at the explicitness game What I was trying to imply is that all the functions I mentioned require explicit in/source and out/destination ranges.

[nick (Nick Shaw)]

Yes, it’s really just CLF’s use of only Min or Max values in the Range node as a special case to perform one-sided clamping, simply to avoid having a dedicated Clamp operator, which introduces the potential confusion, I think.

[doug_walker]

Right, so hopefully my proposal helps since it removed the ability to use the single-sided case to do an offset.

Thanks for the links to the other examples Thomas.

Doug

[jdvandenberg]

" 1. Fundamentally, the spec provides two ways of applying a shift and scale operation: Matrix and Range. The difference is that the Matrix does not clamp and the Range does."

I like such simplicity.

[jim_houston]

In the middle of something else, so have not had time to check the proposal in the thread,

however, I can comment that the Spec is ‘developer-facing’ but mixed with information for a
LUT designer. So I can see where that can get unclear. Doug’s CTF web pages are a much better user
description of the nodes for example.

On the in-depth, out-depth question, the sentence is part of a paragraph, and the whole paragraph is the situation being dealt with (again for the missing end case as a default) not a problem when all four elements are defined:

" If the input and output bit depths are not the same, a conversion should take place using the range elements. If the elements defining an InValue range or OutValue range are not provided, then the default behavior is to use the full range available with the inBitDepth or outBitDepth attribute used in place of the
missing input range or missing output range, respectively, as calculated with these equations:”

[jim_houston]

I agree with Nick’s statement of what the expected behavior, and yes this is mostly
about what happens when either the minValue or maxValue is missing.

The Range Node is about mapping one range to another, but especially allowing
the ability to select a floating range including negatives and map it to a different
floating point range. so a {-1000,1000) should be able to map to a {1000,3000}.
In this case, the scale=1, but the other term is { in + 2000 } to offset to the right place
in the new range. Doug’s modified eqn would just echo the ‘in’ value and clamp it against
the MIN or MAX. This is still the wrong result. Nick’s modification also only focuses on ‘scale’
when both scale+offset are needed.

The ‘special math’ at the end for missing entries does not do this.

I still recommend

A) deleting this ‘missing elements’ section and requiring all four entries and indicating CLAMPs explicitly.
B) creating a RANGEF node that mandates only the basic equation and no missing entries for floating point.

Jim

[jim_houston]

I would note one other small bug…

the rangescaler for integers has a - 1 that should not be there.
the Pure ints 12i/10i 4096/1024 = 4 X for the scaler which is correct.
The minus 1 gives 4095/1023 = 4.002932 which is incorrect.

Jim