I now see Jim’s concern and think I’ve traced the origin of the “noClamp” “style” attribute. I think this was a compromise introduced back in the v2.0 of the spec.
The historical threads jive with the same differences we highlighted on the call, namely - understanding <Range> intended behavior in two different ways:
<Range> is a scale and always clamps
<Range> is just a scale (but not always a range limiting scale - no clamping) - In this context, you could use the node to ‘range’ (or ‘scale’) floating point values from [-0.5, 222] into [0,1] and values such as -1 or 230 would pass through as -0.00224719101 or 1.035955956, respectively.
I propose two (maybe three) solutions, of which I’d like to hear opinions…
If the goal with this spec revision is a more understandable, user-friendly approach, then my suggestion is:
split <Range> and <Clamp> into two separate nodes
rename <Range> as <Scale> since that is how it would behave. Yes, this would add yet another node for implementers but for a LUT creator or user it would serve as a “convenience” function. Behind the scenes in the implementation it would most likely still create a scaling matrix w/ offsets to apply to RGB channels by populating a 3x4 or 4x4 matrix along the diagonal and in the 4th column. A bit more for implementors but perhaps slightly more direct for the user.
If the goal is to simplify the spec for implementors, then my suggestion switches to:
Remove the style attribute and "clamp" and "noClamp" options. Those who want no clamping use a matrix and those who just want to clamp use the range - with either all 4 sub-elements or just a min or max pair of sub-elements.
Clearly state for LUT creators/users that <Range> is intended for uses such as limiting range or scaling to a limited range (e.g. SMPTE legal range).
Instruct LUT creators/users who do not want clamping to occur to use a <Matrix> to achieve non-clamping ‘range scaling’.
For example, we’d show how to populate the diagonal of a matrix with the scale value defined in scale = (maxOutValue - minOutValue)/(maxInValue - minInValue) and then the offset column would be equal to minOutValue - minInValue * scale
A third option would be to leave “as-is” but supplement with better guidance on the effect of the present MIN( maxOutValue, MAX( minOutValue, value)) construct…
Replying to myself as I continue to look at the way the draft was worded…
I have now added back the "Clamp" behavior as a separate operation on the initial scale/offset function, only applied if style="Clamp".
Here’s an Overleaf version with just the Range node isolated for easy reference: Range Node Only
I keep oscillating on this but the more I think about it the more I think we already have it as close to satisfying both camps as possible. I am currently thinking that my favored option is to leave it defined pretty much as-is in v2.0 of the spec but just adding more clarity that the Range node can be used for clamping or just as a simple scale (i.e. no clamping) if you want to, but you need to tell it that. So we sort of already have the user-oriented “convenience” function form, as long as they set style="noClamp".
I haven’t attended the call, i.e. typical NZ TZ conflict, but I would favour explicitness and simplicity: one node, one action thus one node to change range and one node to clamp, it will be then obvious at first glance where and when clamping is happening.
NOTE: The Range element can be used to clamp values
Thanks, @Thomas_Mansencal for offering your opinion on the issue and apologies for always having meetings at conflicting times.
I also don’t prefer the wording (or existence) of that note but it’s pulled directly out of the v2.0 spec (but put into a note in this draft because I was hoping to get rid of it or otherwise address it).
Historically, I think it originated when the original author’s intent for Range to serve as a scale operator needed to be reconciled with a different interpretation that Range was basically a clamp operation. Thus statement “The Range element is also used to clamp values” was born.
There is a lot of history on this topic and many hours were spent by the committee during the writing of the v2 spec back in 2014. Here is my attempt at a brief summary of the relevant points:
Fundamentally, the spec provides two ways of applying a shift and scale operation: Matrix and Range. The difference is that the Matrix does not clamp and the Range does.
When the v2 spec was written, Jim objected to having the Range always clamp and so we added the noClamp style option. However, this is purely a convenience for CLF authors (for example, under the hood OCIO will convert a noClamp Range to a Matrix, functionally they are no different). Also, there need not be any performance hit to using a Matrix since implementors can easily optimize the case where the off-diagonal terms are 0 (as does OCIO).
As much as we all strive to avoid improper clamping, clamping is needed in order to accurately implement certain transforms (including some ACES transforms). So CLF absolutely needs a way to do this.
There was an assertion during the call that Range cannot map [-0.5, 222] into [0,1]. However, that is incorrect, it can do that and furthermore allows the option to clamp or not.
The objections that have been raised seem to be not fundamental issues with the math but more with the “aesthetics” of the design. Please keep in mind that this was a committee effort going back over a decade and that it won’t be possible to please everyone on naming. If we were starting from scratch, I would absolutely support the idea of further discussion about naming and perhaps doing a poll with various options. However, we’re talking about v3 of the spec, and a number of companies have implemented it at this point (it’s also in OCIO now). At the end of the day we want more people to implement this and I don’t think it helps the cause to signal that the syntax is open to being redone at every revision of the spec.
There was some discussion during the call about the formulas for the case where only the min bounds or only the max bounds were specified. We added those formulae in the v2 spec to clarify for implementors what to do if they get a file that does that. However, there is only one important case of that type, which is when one needs to clamp at the low end or high end but not both. If people want to remove those formulae, we could add a limitation which is to say that if a Range only provides one set of bounds (i.e., either min or max), that it cannot shift or scale.
Thank you @doug_walker, for taking time to put these points in writing. I’m sorry we have had to go back yet again to go through all this, especially this late in the revision process and because it’s been discussed before.
I did review the conversations from around the 2014 v2 publication timeframe re: this issue and came to the same conclusions as your bullet points.
I agree with #5 that as of right now the functionality is there to satisfy both parties, it’s hard to please everyone, and there’s no benefit to changing names (in fact it could be detrimental to adoption as it will break existing implementations). Other than for “clarity” there’s no reason to re-segment into separate nodes or otherwise rename the operator. Just keep as-is and make even clearer the functionality is already built in for both requirements - scale and clamp.
I am glad it was brought up so that nothing is overlooked but I think we’re still doing the “best” thing here in terms of servicing the need to offer a “convenience scale/offset” and also a “clamp”, as well as not unnecessarily changing the existing spec just for the sake of change. If we were building from the ground up, yes, we could consider restructuring, but it seems a step too far.
I am proceeding on to finishing the other sections. As of now, nothing has changed in terms of functionality of the <Range> node from v2 to this version of the spec. We had already reached compromise previously and unless there is some yet-unheard-extremely-compelling reason to change, I am in favor of leaving as-is.
I’m not aware of what was said in the past, is it documented publicly? In any case clarity is important. Clamping functionality of the Range node could be deprecated, and a new node introduced without breaking anything.
I’m sorry, but this discussion on Clamping is off point. My dislike was of the use
of implicit clamping an one end but not the other based on whether a parameter was
missing or not. (This is not a good method) Doug brought up the same argument that
because the original spec was implemented differently than what it stated, that we should defer to
the only one implementation. I still disagree with that.
But all of this misses my main objection. The LUT format is intended to be able to create a ProcessLIST that works on full bit resolution files in their native float representation. But the calculation of the (if not present) scale factor ALWAYS set floats to a 0…1.0 assumption (scale=1) But this is only true if the LUT only works in a range of 0…1.0 on the input side and this is EXPLICITLY not so. A range of -0.005 to 222 is a valid float input.
My recommendation is to delete the following text, and always include both MIN and MAX values in the Range Node. There should be no missing float range assumption.
note: the full range available turns out to be a wrong statement (we are not using the full range of floats just the 0 to 1.0 subset. [This can be a poor processing result squeezing the whole float range into a 1.0 container range.)
is to use the full range available with the inBitDepth or outBitDepth attribute used in place of the missing input range or missing output range, respectively, as calculated with these equations:
In the formulae below, if the bit depth is integral, then rangescalar() is defined as:
rangescalar(bitDepthInteger) = 2bitDepth − 1 If the bit depth is specified as floating point, then:
rangescalar(float) = 1.0 If only minimum values are specified, the formula is:
Scott: please review the spreadsheets I did at that time. They were about the calculations
of floats going wrong under the implicit missing item equations.
Since Doug is not presenting both points of view, I am sorry I think I have to responsd in line.
On a ‘lot of history’ – this conversation was never concluded and we ran out of time (which is why the CLF 2.0 spec was never finished and is listed as DRAFT)
Yes, matrix allows scale and offset, but it doesn’t allow extraction of a portion of a float range
and only work on that range. (the RANGE node is for that purpose and it still works. It is the implicit clamping at one end that is the problem – the workaround is to always use all 4 metadata items but the spec should not allow an implicit operation that causes the math to go off.
Again my objection was to the implicit one-side clamping, not about the presence of clamping.
on point #6 and this previous sentence, another solution is to make explicit the clamping behavior
ClampHi, ClampLo when needed. Adding another node with a different behavior is also an understandable approach but it needs to get back to the original intent on handling the full float range (not assuming float=1.0 scale) I propose deleting the objectionable text in this thread. The focus should be on that issue not the general existing of clamping – and no it is not a naming disagreement, it is about the fundamental math.
There should be a note about use of the matrix 3x4 node to scale and offset floats in an unclamped fashion. I agree that Clamping is essential.
I am sorry to say I may not be available until late in the call and request that this discussion
be deferred until I can join.
What if we enforced that when style is ‘noClamp’ (even when only min or max pairs are provided) then no clamp is applied. Would that work? Then nothing is implicit because you are explicitly saying not to clamp.
If we went further and listened to @Greg_Cotten, we could require that style be set, which would require explicitly stating the intended behavior.
Side note: What might be a practical use case for specifying just min or max pairs and not wanting it to clamp?
Yes I did review this but I had difficulty making sense of it for the min/max only cases when mixed with in/out bit depth differences. Perhaps if I had a practical example to help wrap my head around what the intended behavior is rather than just looking at numbers - then I might be able to better grasp the numbers to see some meaning in the current behavior.
For those interested in taking a look, here is the spreadsheet Jim is referencing: range_node_eval_v4.xlsx (53.6 KB)
Hey Jim, it looks like the website messed up the formatting of your message. It’s not clear what you’re objecting to or what your proposal is. As Scott suggested, I think it would help to have a specific example to discuss.
My proposal was to delete the text in the Range node that deals with the assumed scaling behavior if one piece is missing.
An example is for an input of 0 to 10,000 nits and an output of 0 to 1023, the scale factor that gets calculated is (1023/10000) = 0.1023 when all four parameters are present. This is the correct conversion
from a 10000.00 float to 1023. With a 5000n input, RGBout = 5000 x 0.1023 + 0 - 0 x 0.1023 = 511.5 the right answer.
Exactly per the spec, if only the max values are specified, the scale parameter becomes ( 1023/ 1.0 ) or 1023 X as a multiplier. This is 10000 Times OFF. This is the damage from assuming all floats operate in the 0 to 1.0 range.
With an RGBin of 5000n and following the spec exactly with the equations as specified for the max only case…
RGBout = MIN(1023, 5000n x 1023 + 1023 - 10000 x 1023) or MIN ( 1023, -5113977 ) gives the wrong
number in the full float input situation. (The 511 is there but just of completely the wrong magnitude).
My proposal is to delete rangescale(float)=1.0 and everything after it and require all 4 parameters with a specific style addition of ClampHI or ClampLO. This minimizes the changes to the node input but if it still seen as necessary to keep the damaged math of the RANGE node as is, create a new RANGEF node with a note about the assumption in the existing RANGE node.
Thank you Jim. When I plug in your example values I get a different number. But in any case, I’m open to prohibiting the use of only one side (max or min) in a case that does not allow a simple clamp.
It took me a little while, but I now follow your example, @jim_houston, and get the same result you do. I did not realise initially that your scale=1023 was coming not from maxOutValue=1023 but rather from outBitDepth=10 which is implicit, but not explicitly stated, in your example.
It does seem to me that the maths in the current spec is just simply wrong (or at least counterintuitive) for the single ended clamping cases. I have (I believe) a simpler, float only, example which illustrates this.
maxOutValue and maxInValue are not included at all in the calculation of scale. So if inBitDepth and outBitDepth are both 'float' then:
If I then set maxInValue=2.0 and maxOutValue=10.0, I might expect the operation to simply multiply input values by 5, and clamp at 10.0. Therefore 0.5 input would produce 2.5. However, testing this in Python, I get:
It is effectively using implicit values of minInValue=0.0 and minOutValue=8.0 which is not what I think most people would expect. Intuitively you might expect 0.0 to be used for both.
Indeed in @jim_houston’s example for the result in the second case to match that in the first, implicit minimum values of zero would have to be used. But in fact an implicit minInValue=9999.0 is used.
On a separate subject, I am not particularly keen on this sentence in the spec:
“If the input and output bit depths are not the same, a conversion should take place using the range elements.”
Who “should”? Does it mean that people creating CLFs should add e.g. an explicit range mapping from 0-1 to 0-1023? Or does it mean that CLF implementers should apply bit-depth based range scaling automatically? It kind of means both, which is a bit confusing.
If a CLF creator includes a Range node with inBitDepth="32f" outBitDepth="10i" if they provide only, for example, minInValue=0.0 and minOutValue=0, a normalised float to 10-bit scaling will be automatically applied (with negatives clamped). If they provide all four range values, they need to be values which correctly apply that scaling, and inBitDepth and outBitDepth are ignored in the calculation of the mapping.
