ACES Retrospective and Enhancements

Dear ACES Community,

On March 9th a paper titled “ACES Retrospective and Enhancements” was posted on the 3D-PRO email list by Thomas Mansencal, Sean Cooper and Kevin Wheatley. It is clearly the result of a substantial effort by the authors, along with their contributors and reviewers.

As we consider the next round of development for ACES, this paper could not come at a better time. With permission from the authors we are re-posting the paper here for review by the ACES community, and we solicit your feedback.

Thank you – the ACES team

PS You can reply to this post with comments or if you want to focus your comments on one area, please create a new topic in the Feedback and Requests section and put “RAE”: (your specific area of comment i.e. ACESClip)" in the subject line.

aces_rae_2017.pdf (129.9 KB)

2 Likes

Hi Steve,

Thanks for publishing the document, glad to see the Academy officially acknowledging the paper. Eager to see what is next now :slight_smile:

I will paste here the summary of the paper:

In summary of the various requests, proposals and questions that were addressed in the document the authors, contributors, reviewers, and members of the ACES community wish that:

  • The past, current and future of ACES development will be more open to peer-review and academic discourse.
  • A public record of any experimentation, analysis, and implementations, along with useful data (image datasets, viewing conditions, display attributes and observer physiological characteristics) will be initiated.
  • The CTL codebase documentation, especially the unexplained constants will be revised.
  • A document on ODT creation akin to P-2013-001 [Amp15] will be written.
  • The CTL codebase and the official ACES OpenColorIO configurations, along with relevant documents, e.g. technical bulletins, will be stamped with DOIs.
  • The CTL codebase reference implementation will adopt a solid unit tests suite while checking the ACES OpenColorIO configuration state against it.
  • The Academy will promote a system for vetting and integrating IDTs with user feedback.
  • The Academy will strongly promote CLF adoption by its partners and the community.
  • The Academy website will be the host for best practice guides for usage of ACES and ACES OpenColorIO configurations within DCC applications.
  • The RRT will adopt a more neutral look and a faster parameterized mathematical implementation that will allow better control over the tonescale, thus contributing to spread of RRT usage and enabling true archival of creative intent.
  • The RRT will be made fully invertible, and that the various sweeteners will be moved to a default LMT.
  • The ODT will be parameterized to adapt to a wider range of displays and viewing conditions while reassessing usage of CAMs to account for induced changes of the perceptual correlates by the luminance increase of the HDR displays and different viewing conditions.
  • An officially publicized and documented workaround for highly saturated emitters artifacts will be available in the CTL codebase until a long-term solution is developed.
  • The ACESclip file format will be simplified and advertised, as the archival promise is currently broken without proper metadata. XMP would be useful to consider along storage of the ACESclip file within the ACES container.
  • Remaining issues in the ACES OpenColorIO configuration such a visual mismatch and ICC profiles clipping will be addressed.
  • The ACES OpenColorIO configuration repository will be taken under the Academy wing and its development will be more directly supported by the Academy team.
  • The ACES OpenColorIO configuration builds, i.e. the config file, LUTs and ICC profiles, along the CTL transforms will be hosted as first class products on the oscars.org website.
  • A robust solution will be found for integer based applications such as Adobe Photoshop.
  • The broadened scope of ACES beyond its original context will be accounted for with Video Games being a strong adoption driver to be reckoned with.
  • Discussions regarding potential changes to the reference environment will be held.

The Github repository with the Latex source code is available here: GitHub - colour-science/aces-retrospective-and-enhancements: ACES - Retrospective and Enhancements

Bests,

Thomas

3 Likes

This was posted by Ray Feeney as a response to the original post on 3D-PRO. Re-posting here for convenience.

Thomas, Kevin, Sean (and the rest of the contributors),

First, I just want to express my sincere thanks for this paper. The amount of work, the clarity, and the usefulness of what you have created cannot be overstated. And I personally don’t disagree with anything raised as issues – they exist, they are real, and they need to be addressed.

And yes, ACES is an open source project so input (and participation) is actively encouraged. Hopefully your team (and others on this list) will take the time to go to community.acescentral.com and participate in a discussion around these topics. The paper is now posted in the discussion section under a new category titled: Feedback and Requests.

It’s pinned to the top of that category with instructions on how to join the conversation and launch discussion threads or topics.

It was just posted so there aren’t any conversations yet underway, so I think I will say a couple of things here to try and spur some thinking and inspire some participation on community.acescentral.com. And, of course, if anyone wishes to discuss anything offline, feel free to reach out to me.

As Thomas mentioned, the ACES leadership is changing. The Academy has some very good rules about term limits on committees and projects. Fresh voices are always encouraged. As ACES ramped up to a V1 official release, these term limits were waived in the interest of continuity. Now that conversations about post V1 steps are being discussed (including a ramp up to a V2 that can hopefully address the known issues), new project chairs will be selected. One has already been publicly announced – Annie Chang of Marvel/Disney.

So please take the below comments as my personal opinion – I am no longer speaking officially for ACES or for the Academy. But, obviously, I care a great deal about the next steps and I hope that I can inspire some of you to actively participate going forward.

The paper covers a lot of specifics, but here I think I will simplistically divide things into three primary categories – The processes of project participation and transparency, the gamut mapping issues of IDT input mapping and related artifacts around bright emissive sources (like colored lights, LEDs, or signs), and the general rendering transforms (in particular, the RRT). All three should inspire vigorous conversations on acescentral (and perhaps additional key topics in the paper will surface also).

When the ACES project started, there was no support for many of the underlying concepts in any of the typical commercial tools. Which meant that in order for anyone to actually see any ACES images in a properly vetted and controlled manner, people needed to come to the imaging lab at the Academy or presentations in the Dunn theater. Over fifty companies supported the efforts of their employees to participate in ACES development and to attend the monthly meetings in Hollywood. Companies like (for

example) Kodak, Fuji, and ARRI sent representatives thousands of miles each month to participate, but the natural fallout was a preponderance of local participation. Meaningful high speed internet remote participation was not an option a decade ago. The world is now a different place and I think this paper is an existence proof that important, useful, and influential ACES work can now take place anywhere on the globe. ACES development is primarily driven by volunteers who are facilitated with some amount of administrative and engineering help from the Academy. Rightly so, the paper calls for a reevaluation of how the broader community can participate in ACES development in a more traditional open source style model. How best to accomplish this transition is certainly worthy of conversation with the ideal goal of more directly empowering interested credible participants around the world. So please go to acescentral and discuss.

Second, the implications of an IDT type workflow seem to be completely validated by the industry. The concept of bringing all elements into a common working space seems to no longer be the least bit controversial.

Even systems that do not subscribe to the balance of the ACES components have adopted the idea of using an Input Device Transform into a common workspace of some form. Although many imaging devices and techniques work well for the vast amount of naturally occurring reflective colors, our world is everyday becoming more populated with active screens, brightly and highly saturated near monochromatic lights, and incredibly wide gamut signage. New techniques for gamut reduction into some form of working space (without the typical edge case artifacts) will need to be pioneered and evolved (with or without ACES). It is very valid work that needs to happen and it seems to be appropriate for a focused open source development effort. All ideas are welcome. Once again, please go to acescentral and discuss.

That brings us to the issues of the current rendering transforms. The vast majority of that discussion belongs on acescentral, but I will say a couple of things here. When ACES development started, the de facto workflow was 10 bit DPX log based. I know that it may come as a shock to many of you in the VFX part of the industry, but there was a great deal of resistance to a 16bit half float linear workflow as an underlying mechanism to bridge the silos of tasks in motion picture creation and post production. Granted that in the DI process, native camera data is still usually placed on the timeline and transformed on the fly through an IDT into a common working space, but in general, the world is no longer afraid of the linear light workflow concepts embodied in an OpenEXR workflow (even if parts of the industry are still resistant to storing the data in the OpenEXR container itself). ACES is responsible for socializing the vast majority of that change in thinking.

Prior to ACES V1, there was a great deal of discussion in the ACES working groups about what would be a “bridge too far” for ACES version one. There are two famous slides in the early ACES presentations that were created by Josh Pines. The first was “How you work today” and it had all the DPX terminology and transform boxes laid out in a diagram. The second was “How you work in ACES” and it was identical except that the boxes had different legends that corresponded to ACES terminology. There were three components in ACES that did not appear on that slide because there were no corresponding boxes. They were: Look Modification Transforms (LMT’s), the Common Lut Format (CLF) and ACESClip (the metadata container to specify the transform chain). All the other parts of ACES overlay on existing mechanisms so the effort for developers to add support for these overlay ACES components was deemed manageable. In fact, most ACES implementations support only those overlay functions and have missing (or incomplete) support for LMT’s, the CLF, or ACESClip. As a working group, we agonized over the RRT. As this paper points out, a serious discussion of the benefits of separating the basic rendering from the parts that might be characterized as “look components” (and breaking the RRT into a LMT and a simplified rendering that would be more easily dealt with for alternate looks), is a hugely appropriate conversation. Rightly or wrongly, there was a sizable contingent of the working group who felt that shipping a system that required valid LMT support in order to function at all, would have been a mistake. So we have the RRT as it stands in V1 – which allows for fairly simple modifications to tools and pipelines to support ACES and for many people it has been a success. Everything brought up in the paper about the RRT is correct and valid – but it is hard to break the RRT down into a LMT and a rendering if there is no support for LMT’s in the commercial tools.

We used to ask the community to ask vendors “When are you going to have ACES support in your tools?” Now we are asking the community to query vendors “When are you going to have meaningful support for LMT’s, CLF, and ACESClip in your tools”.

There is an interesting thing that has happened, though. Because we only had the option for one box in ACES V1 for the components that were rolled up into the RRT, there have been a significant number of high-end industry projects that have used all the ACES components but have used a custom rendering – their own “alternate RRT” that better addressed certain specific project requirements. So, in my mind, it is appropriate to separate the discussion of the architecture of the ACES system from the specifics of the actual rendering transforms. As is pointed out in the paper, a fairly large number of projects have used the ACES system components very successfully – but with a substitute rendering tailored for either a particular look or a style of working. I would claim that this level of use pretty dramatically demonstrates the robustness of the system architecture. I am sure that people are aware of the significant number of shows that have been supported by Bill Feightner at Colorfront or Josh Pines at Technicolor using ACES components along with some very interesting and robust rendering transforms.

Something like seven or eight of the top 10 grossing movies of the last couple of years have used ACES system components coupled with either a custom rendering or the RRT.

The paper calls for the splitting of the RRT into two parts (a LMT for the look modifiers and a simpler RRT). This implies a need for LMT support, the Common Lut Format to express the LMT transform, and ACESClip to carry the information as to what transforms have been cascaded together to form the actual total system rendering. There are a significant number of ACES development participants who agree with that recommendation. It isn’t accidental that these three necessary components are contemplated – and that they exist as draft proposals. It seems to me that it would be extremely valuable to have a conversation (on acescentral) around the questions of “Is this the correct thing to do? Are the draft specifications for the pieces needed to support this correct, sufficient, and implementable? And, will such a refactoring actually make ACES truly what it needs to be to fulfill on its potential?”.

As we contemplate an ACES2.0 (along with a cohesive roadmap for ACES system components), broader industry participation is invited and encouraged. It is open source – come help us flesh out the missing pieces and tune up the elements that need to be improved. As Annie Chang has said: “ACES is a thing now – not just a concept.” ACES is not going away. If there are parts that need work, then help us address those shortcomings.

Please help us understand how best to engage with with the production community to productively move ACES forward. I think this paper is an incredible first step – and one that has been very well received by the ACES leadership team. Again, thank you – I know how much work goes into such an endeavor.

I hope that this post is helpful. There are a lot of people on this list who are interested in ACES and I may pop back in few weeks with a status summary. Feel free to comment here, but please at least cross post your thoughts to acescentral (and hopefully ignite a sustainable conversation on “ACES next steps”).

Thanks,

Ray Feeney

1 Like

Speaking as a colorist, specifically to the RRT, I would very much appreciate the “look portion” of the RRT either being removed, or maintaining it in a separate component in a user definable LMT. As a predominantly commercial colorist, I feel like I am being forced into a print look that is not desirable on most of my projects.

2 Likes

Counter balancing to the desired level you need does not work out for you?

@jafalcon: Thanks for your feedback!

It was one of the main point of the paper. I think all the people taking part in its creation agreed that the default look should not get in the way of artists, i.e. it has to be the most neutral possible, with sweeteners moved to a factory LMT like you underlined it.

@Margus_Voll: Contextually, e.g. realtime engines, it is very hard to counter balance, especially the black and highlight rolloffs: not all the applications implementing ACES provide grainier control enough to counter balance the default look nor they do offer LMT support as it is an optional component of the system.

Cheers,

If you are happy with it we can spawn a new thread just dedicated to the RRT in this sub-forum.

Thomas

2 Likes

I recently reached out calling for a series of still-missing components ― whose original concept I designed in 2013‒2014 during the pre-release stages leading to ACES 1.0, More specifically, they are:

  • Improved integration of ACES in IMF and other mezzanine, mastering and long-term archival formats; in particular, ACES-aware OPL syntax.
  • Improved consistence of metadata/tags layout in OpenEXR and MXF containers, including pictures in colorspaces different from ACES2065-1 (usually, ACEScg). This means improving on from SMPTE ST2065-4 and ST2065-5, possibly embracing other production containers, such as QuickTime and ARRIRAW.
  • Redesign of the ACESclip XML metadata tool to sidecar with non-EXR files as well and support full stack of production metadata
  • Encourage the use of CommonLUT format (CLF) to increase its read/write support by more software applications
  • Include a “Color Pedigree” (in both IMF, SMPTE2065-4, SMPTE2065-5, as well as in the ACES clip container) that tracks the “past” and “present” color history of a clip: how it was generated and manipulated to become the clip the pedigree is currently addressing, including optional intents for output colorimetries. Procut partners honouring such color-pedigree may better manipulate the color management of such pictures – even pictures not (yet) in ACES colorimetry, like raw-camera pictures or assets not sourced in ACES, for a correct interpretation in an ACES pipeline.
  • Add color-pedigree metadata in the CommonLUT format as well, so that LUTs are described by XML tags as well, for applications to better acknowledge its use and possibly warn users about (possibly) wrong applications (e.g. converting from float-to-int and then back, or applying a non-invertible transform, or using a missing or wrong shaper-LUT).
  • Agreeing with the ACES Retrospective and Enhancements document, analytical invertibility of the RRT is a key requirement for the improvement of a truly interchangeable framework in diverse M&E echosystems.
  • Better integration of UUIDs for ACES CTL components like IDT, ODT, RRT, LMT and generic utilities so that, for example, concatnation of color operations can be recorded (and stacked) and, in case of successfull juxtaposition of a color mapping and its inverse, simplifications can be applied to reduce approximation/roundoff errors. Academy-original or other proprietary CTLs can have optional hashing syntax at the beginning for their integrity self-check.
  • Creating a CTL repository that includes Academy-official ODTs from various ACES releases and vendors (thus also IDTs) and a way to automatically download and validate these CTLs against their UUID. GitHub may be a choice, but an oscars.org domain would be better. This should work in a way similar as to how each Digital Cinema manufacturer hosts its servers’ public certificates (retrieved by the servers’ serial numbers as unique IDs) – but without the need for all that security.
  • Official documentation that maps individual metadata and tags from non-ACES production file formats, into both ACESclip, SMPTE2065-4 and ST2065-5 metadata/tags, to use them when addressing/referencing non-ACES images (such as REDCODE, ARRIRAW, DNG, MXF, ProRes, XDCAM). For example:
    • which unique identifiers or tags from those file formats, etc. should be used as picture UUIDs
    • which TimeCode fields should be considered
    • which colorspace mappings should be used for ACES colorspaces and/or AP color primaries’ families
  • Consider creating an integer-math encoding, “ACESproxy-variant” of ACEScct colorspace, for on-set viewing and CDL of material to be graded in ACEScct.
  • Extend the UX Guidelines to OpenColorIO and greatly improve OpenColorIO naming-convention to better match with ACES (e.g. remove ambiguities such as “Utilities - Texture” in an ACES naming context)
  • Relevant for tape archives: include / complement ACESclip metadata with LTFS specific metadata to be included in the LTO Index partition for long-term archival of color- and production-specific metadata that is really application-agnostic.

Not all of the above are high-priority, but necessity for all of them was asynchronously confirmed by the several teams I worked with in the last few years ( post/finishing, CG/VFX, on-set/D.I.T.) in several professional applications of ACES 1.0.x for theatrical releases. More specifically, I think these missing or yet-incomplete components are:

1 Like