Why 'Utility - Raw' as 'reference' for ACES1.0.3 and not ACES2065-1?

abraham.schneider · April 13, 2018, 3:18pm

While reading through the official 1.0.3 config file and trying to understand all bits and peaces, I often come across the ‘reference’ role definition. In the official OCIO docs, the ‘reference’ role is said to be the ‘heart’ of the whole config, so all the ‘from_reference’ and ‘to_reference’ transforms are working with the colorspace defined with this role, right?

So why is it not pointing to ‘ACES - ACES2065-1’, which in my opinion is the ‘heart’ of the ACES world. And why has ‘ACES - ACES2065-1’ different definitions than the ‘Raw’ definition (isdata false vs. true; allocation, etc.).
I always find this very confusing, especially if you try to explain the concept of ACES and OCIO.

Thanks, Abraham

hpduiker · April 13, 2018, 3:52pm

Hey,

The first colorspace defined in the config is the ‘reference’ that the other colorspaces’ to_reference and from_reference transform lists go to and from.

The ‘reference’ role is a little bit ill-defined but, I think, is supposed to be used for reference photography. When we were choosing the colorpsaxes foe the roles, choosing Utility - Raw for the reference role was mostly a pass on making a decision.

The two uses of ‘reference’ are an unfortunate coincidence. Note that none of the roles are required to have a valid config.

Hope that helps,
HP

abraham.schneider · April 13, 2018, 7:55pm

So it’s the order of the colorspaces that defines which one is the ‘master’? Wow, that’s completely new to me. In the OCIO docs I could only find the sentence:

role:
reference - the colorspace against which the other colorspaces are defined

Now I’m really confused. Shouldn’t that be pointed out really really well in the OCIO docs, which colorspace is the ‘master’? (Of course that would be a request for the OCIO guys, not here for ACES).

Thanks!

hpduiker · April 13, 2018, 9:23pm

Look a little further down on the page.
http://opencolorio.org/userguide/config_syntax.html

“
First the lnf colorspace (short for linear float) is used as our reference colorspace. The name can be anything, but the idea of a reference colorspace is an important convention within OCIO: all other colorspaces are defined as transforms either to or from this colorspace.
“

The note on the reference role is confusing though. I’ve wished for a while that roles acted more directly like colorspaces. In practice, they are only labels.

Perhaps the next iteration of the config should still make the reference role equal to the reference space.

HP

ChrisD · April 14, 2018, 10:29pm

Hi @abraham.schneider,
Now that you’ve got the “reference color space” concept clarified, I wanted to mention one other thing.

Something that can be frustrating when setting OCIO configs is that the implementation (or lack thereof) of the “roles” feature is different from one host application to the next, and those implementation details are often undocumented. Sometimes the role assignments are mapped to user settings or operations in the app (e.g. Nuke).

Other times the role color spaces affect the image processing and the user may not be aware. An example might be rendered image data being converted to the “rendering” role color space when output files are written, even though no explicit GUI setting is given to control that behavior. Anyway, keep your eye on those role assignments if you are troubleshooting the OCIO behavior in a host app.

-Chris

Tobias_Pfeiffer · April 15, 2018, 5:13pm

Hi,
whenever I am modifying the default ACES OCIO config, I am putting new colorspaces at the top. If the first colorspace of the config would define the reference, these modified configs wouldn’t work anymore. But this is not the case…
My initial guess would be that the reference space that is used for “to_reference” and “from_reference” conversions is the one that is simply missing these parts.
Would be interesting to see what happens if more than one colorspace meets this criterion?
Does this make sense?
Best,
Tobias

abraham.schneider · April 16, 2018, 7:30am

Interesting idea, but in the ACES 1.0.3 config there already are at least two definitions that meet that criteria: ACES2065-1 and Utility-RAW. So which one is used?

So it seems like ACES1.0.3 config is working, but no one knows why/how!?

abraham.schneider · April 16, 2018, 7:43am

In practice, I think it doesn’t really matter, as all colourspaces without from_ and to_ definitions would be ‘the same’, but I’m wondering because they were defined with different allocation params, so there should be differences technically when rendering on the GPU, depending on which one of these colourspaces will be used by OCIO, right?

dithermaster · April 16, 2018, 8:19pm

With all due respect to hpd, I don’t believe that is the case. I’ve looked at the source code, and I don’t see anything special about the first colorspace. I do see that the role called “reference” is used when looking up the to_reference and from_reference transformations. So I believe that the reference role is what determines the reference color space (not the first colorspace listed in the config).

///d@

dithermaster · April 17, 2018, 3:07pm

Correction: That’s how our host code calls OCIO. We ask for processors between color spaces and ROLE_REFERENCE (on the way into working color space), or the other way around (on the way out).
OCIO itself does not use ROLE_REFERENCE for anything special, but host apps typically do. You can also ask for processors between any two color spaces and OCIO will implicitly use the “to_reference” and “from_reference” paths to create a processor (including optimizing out redundant steps). It doesn’t use ROLE_REFERENCE to do that (like I thought it did). Still, the “first” color space is not special in any way. All that really matters is the “to_reference” and “from_reference” definitions, and if hosts do use the ROLE_REFERENCE for their working color space, it should ideally be set to the same color space that those transforms use.

alexfry · April 18, 2018, 1:47am

It’s been a while since I’ve hand rolled a config, but I always worked under the assumption the first listed colorspace was the primary/reference colorspace that all others were defined relative to.

I always read ROLE_REFERENCE as "photographic or art reference material’.

hpduiker · April 18, 2018, 5:40am

Based on the posts on the OCIO list, it looks like neither the reference role nor the first colorspace actually formally specify the ‘reference’ colorspace, despite the web page documentation suggesting that both do so. The reference colorspace is apparently only specified indirectly by the config author, in that the to_reference and from_reference transforms all go to and from a common space. In the case of the ACES config, the colorspaces were all designed to transform to and from ‘ACES - ACES2065-1’, which just happens to be the first colorspace.

Given that some apps depend on the ‘reference’ role also pointing to a colorspace that matches the implied reference colorspace, we should probably change the config to do so.

Pretty interesting.

mplec · April 18, 2018, 3:43pm

I may have misinterpreted HP’s comment about the first colorspace but I took it to refer specifically to the aces config Abraham referenced, not to how OCIO configs work in general. As Dithermaster said, all that really matters is the “to_reference” and “from_reference” definitions, since reference is implicit in them. Setting the reference role is a good way to make the intent clear, though.

This conversation gave me an intense sense of deja vu and I just found why:

https://groups.google.com/forum/#!topic/ocio-users/V3oZmjkcfTU

Alex your read on it probably came from what was originally in the web site text, but the only opinions offered up at the time of that discussion were that it was intended as an identifier for the space that to_reference and from_reference implementations convert to/from. And even the spi-anim and spi-vfx configs had reference set to “lnf”, which wouldn’t likely work for art reference, so it does seem like the intent was always to identify the to/from reference space.

zachlewis · April 18, 2018, 5:09pm

Good to see you Matt
Unfortunately, at the end of the day, it seems inevitably to be a matter of vendor interpretation… it’s really a shame about the ambiguity…
HP, if I recall correctly, you once put together a pretty comprehensive review of applications that supported OCIO at the time… I have no idea where I saw that, and I’m sure that hasn’t been maintained (has it?), but I’d gladly contribute to a group effort to detailing vendor support for OCIO and ACES in a wiki or something… that would be a pretty damn helpful resource,

hpduiker · April 19, 2018, 4:23am

The document you’re thinking of, detailing ACES support in various applications, is here (read only):

If anyone is interested in updating the document for more current versions, let me know. I’d be happy to provide an editable link. Just message me.

For those interested in implementations, I think the Nuke nodes are supposed to be a reference. The OCIO Colorspace node creates the path from one colorspace to the next with this call
https://github.com/imageworks/OpenColorIO/blob/master/src/nuke/OCIOColorSpace/OCIOColorSpace.cpp#L207
m_processor = config->getProcessor(context, inputName, outputName);

which leads to this call to create a color space conversion

github.com

AcademySoftwareFoundation/OpenColorIO/blob/454f5fef6e991e645401341cd6fab41c2451b8cd/src/core/Config.cpp#L1357


    if(!src)
    {
        throw Exception("Config::GetProcessor failed. Source colorspace is null.");
    }
    if(!dst)
    {
        throw Exception("Config::GetProcessor failed. Destination colorspace is null.");
    }
    
    ProcessorRcPtr processor = Processor::Create();
    processor->getImpl()->addColorSpaceConversion(*this, context, src, dst);
    processor->getImpl()->finalize();
    return processor;
}


ConstProcessorRcPtr Config::getProcessor(const char * srcName,
                                         const char * dstName) const
{
    ConstContextRcPtr context = getCurrentContext();
    return getProcessor(context, srcName, dstName);
}

processor->getImpl()->addColorSpaceConversion(*this, context, src, dst);

which eventually leads to this function

github.com

AcademySoftwareFoundation/OpenColorIO/blob/454f5fef6e991e645401341cd6fab41c2451b8cd/src/core/ColorSpaceTransform.cpp#L180


                                           const ConstColorSpaceRcPtr & csb)
    {
        std::string a = csa->getEqualityGroup();
        std::string b = csb->getEqualityGroup();
        
        if(!a.empty()) return (a==b);
        return false;
    }
}


void BuildColorSpaceOps(OpRcPtrVec & ops,
                        const Config & config,
                        const ConstContextRcPtr & context,
                        const ConstColorSpaceRcPtr & srcColorSpace,
                        const ConstColorSpaceRcPtr & dstColorSpace)
{
    if(!srcColorSpace)
        throw Exception("BuildColorSpaceOps failed, null srcColorSpace.");
    if(!dstColorSpace)
        throw Exception("BuildColorSpaceOps failed, null dstColorSpace.");

BuildColorSpaceOps
which takes two colorspaces and concatenates the source colorspace’s to_reference (or inverse from_reference) transforms with the destination colorspace’s from_reference (or inverse to_reference) transforms.

So… the code is pretty clear. The reference colorspace isn’t the first colorspace. It isn’t the space pointed to by the ‘reference’ role. It’s the space that the config colorspace’s various to_reference and from_reference transforms use as a target or as a source. Hopefully all the colorspace’s in the config agree on that definition!

abraham.schneider · April 19, 2018, 12:08pm

So technically it doesn’t matter, but if you want to explain ACES and/or OCIO to someone, it would really help to have the roles behave ‘logical’, so the reference role points to ACES2065-1 like explained in the OCIO docs.

One thing that I’m still interested in is, why the two ‘identical’ definitions (ACES2065-1 and Raw) have different allocation parameters. Why do you want to use a log2 curve for a linear colorspace?

hpduiker · April 19, 2018, 2:39pm

Allocation variables are primarily used in the GPU path, when needing to construct a 3D LUT to represent a set of transforms. Using a linear encoding for a linear HDR colorspace will get you pretty quickly into quantization land. Using a log encoding avoids that.

Check out the docs on allocation vars for a longer form explanation:
http://opencolorio.org/configurations/allocation_vars.html

As for the value of the reference role, it’s probably worth updating. It would really be best to update the documentation so there aren’t two different places called out as ‘the definition of the reference colorspace’, when neither is, in practice.

ChrisD · April 23, 2018, 3:24am

Totally agree with the conclusions of @dithermaster and @hpduiker. The OCIO source code doesn’t enforce anything special with regard to the “reference” colorspace. However, host applications may make assumptions with regard to the 1st config colorspace, or the “reference” role colorspace, or both. Keep in mind that this can be an intended application of OCIO, but this behavior may not be universal across host apps.

FYI for anyone using the OCIO nodes in Nuke:
Some operators use the “reference” role colorspace in the OCIO config when converting to/from the node’s “working space.” Even if you set color management to OCIO, and your Project Settings working space to the colorspace of your choice, you may see incorrect or undesired transformations applied in nodes such as OCIO CDLTransform. If you’re using an ACES OCIO config with Nuke, I’d recommend changing the “reference” role colorspace to “ACES - ACES2065-1”, which should keep the OCIO nodes behaving as expected.

I’d love to hear about other people’s experience with this!