Hi to all,
I was wondering if there was any plans to have a dedicated archiving format for a future version of ACES. I’m pushing hard my clients to archive their movies in a future proof “environment” . But OpenEXR are just to “heavy” to be used.
The industry in Quebec is not yet wanting to go down HDR etc… But it would be great to be able to offer an option to do a trim pass at a later date.
Thanks!
At the moment, ACES .exr is the archival format. Are you asking about a compressed
form for TV or other use? It would be great to have feedback on what you are looking for in
an archive file if not HDR bound.
I would add to this that the ACES archiving format is nearing completion at SMPTE: ST2067-50, now in final ballot, specifies an Interoperable Master Format (IMF) package that contains ACES frames. As Jim notes, individual ACES frames are constrained OpenEXR files (see SMPTE ST2065-4). A sequence of ACES frames are wrapped in MXF (see SMPTE ST2065-5), and the wrapped frames are packaged up with audio and other assets in the IMF package for deposit in an archive.
More documentation on the ACES archiving application will become available once the SMPTE standardization process completes, hopefully by the end of the year.
Thanks guys! I’m really talking about having an archive for a film (that might need to be retro graded in HDR). Having a bunch of EXR frames is hard to manage for our smaller industry.
@amaltz: are you saying that we’ll be able to wrap EXRs into an IMF with sound? That for me would e a step in the right direction.
The only problem is that most people shoot their movies on Alexa Prores 4444XQ files. It’s hard to convice them they need to get an ACES archive that will make the final archive file grow 10 folds in disk space.
I guess there’s no solution. We need this to be uncompressed and floating point…
Thanks!
Hi Charles -
Yes, that’s exactly what you will be able to do. The idea is to package the final graded (“baked”) ACES files along with the metadata required to properly display them, e.g., ACES version and expected ODT. The follow-on work will be to add support for specifying LMTs and a few other archiving use cases.
Archiving camera original footage is one of those use cases that hasn’t been specified yet, but we’re always looking for volunteers to take on projects like this!
-Andy
Heya Andy – out of curiosity, what would you say the relationship between ACESclip and IMF is looking like, these days?
Thanks Andy! Please define expected ODT. The point of ACES is to be able to change the ODT to whatever new standard… Correct?
So the expected ODT would be the one used in the “master” grade?
Thanks!
Yes, the “expected ODT” is the one used in mastering and therefore for any subsequent rendering of the archived ACES files.
Changing the ODT might be done for a number of reasons, but most likely to repurpose archived content for another display type and/or viewing environment. And changing the ODT might very well require another color correction pass, ranging from a simple trim to something more substantial depending on the target display and viewing parameters.
-Andy
Hi @chuckyboilo.
First of all the SMPTE standard ST2067-50 “IMF Application #5: ACES” ―which we almost completed and will soon be published― specifies how an ACES IMP (IMF package) should be encoded. As Andy said, it will report information about ACES version and the ODT (filenames for now, but it may be their TransformID in the future).
Multiple ODTs may also be specified in a IMP (e.g. for content to be mastered in different displays and/or dynamic ranges, as per your use-case); technical means to visually verify that the used ACES pipleine is correct are also included within the IMF specs from that standard.
This is because we designed this not just for studio interop, but for long-term archival as well; this is why it internally uses OpenEXR encoding from ACES standard ST2065-4.
As for storage footprint concerns, the ACES IMP is expected to be huge, as the archival copy needs be, first of all, of maximum quality and future-proofed.
That is again why, as @jim_houston said, uncompressed OpenEXR was chosen, while Apple ProRes and Avid DNxHD/R are, instead, both compressed and propietary formats.
ACES IMF uses the usual ACES MXF container (structured as per ST2065-5), where each .mxf video file wraps a sequence of EXR frames (structured as per ST2065-4).
This has been discussed here and there and I’m quite keen to put that back on the table: OpenEXR has a few lossless compression schemes that could be used to reduce the size of the files. I would be keen to have reinstated on ACES Central the reason why one of the scheme was not adopted and maybe reassess if it is a good choice to not allow compression at all in the ACES container.
Cheers,
Thomas
I agree with you Thomas: at least one mathematically-lossless compression scheme should be added.
I believe that the rationale behind the uncompressed choice (dating back at Standard ST2065-4 from 2013), was the poor support, by applications available at that time, of the OpenEXR file format as a whole and, in particular, of a uniform lossless compression algorithm in the EXRs ― at least from the perspective of systems capable of reading and uncompressing images for direct real-time playback.
Today, instead, ZIP and RLE lossless variants are common to most EXR engines.
That would definitely help for small companies like the one I work at!
A little additional history here: when the ACES Project Committee prepared to take the ACES subset of OpenEXR to SMPTE for standardization (ST 2065-4), the group seriously considered including the various compression schemes. To do so meant it needed to come up with a compression “plug-in architecture” for the standards document, as well as taking on standardization of the compression schemes themselves. That was more than could practically have been done at the time, so the Committee decided to start with the basics: uncompressed ACES2065 image data (ST 2065-1).
The intention was (and is) to add compression support in the standards suite. Any volunteers willing to take this on?
I would love to help with that. This can be done for the compression algorithms that are currently published as standards, but this would practically help only if they match with the algorithms that the VFX/post community is using when dealing with production EXR shots.
Andy, we had the same problem when drafting ST2067-50 (IMF Application #5: ACES). Since there doesn’t seem to be a complete (and standardized) registry of (output-referred) color spaces, color gamuts, color encodings, it was not possible to refer to mastering color-spaces in the IMF packages by just mentioning them as metadata in the XML, so an ACES Output Transform’s CTL name (TransformID) is all that could be used in the Standard document.
There is some SMPTE UL used for a very narrow set of color-spaces to be used in certain MXF specs, but this is largely insufficient compared to today’s increasing color-jungle, These ULs should be constantly expanding along with mastering colorimetry – which inflates today at a faster rate than standardization can keep up with.
I think it would also be beneficial, as I said many times in past occasions, that the Academy puts up a public registry about ACES names/values, TransformIDs for ACES core components, and that is standardized somehow so that it can be referenced by other documents (like SMPTE families ST2065 and ST2067). Things to put up in the gesitry would ideally be:
S-2014-002).ctl files may just be linked, in this registry, back from the GitHub CTL repository).As regards CTL hashes and other UUID proposals I will come up in a different topic.
Hi Zach - apologies for the seriously delayed reply. Somehow I missed your post.
ACESclip and the IMF packaging spec are connected, albeit loosely. ACESclip is intended to be the metadata carrier that is “born” on the set and travels with the content through the entire production pipeline (with all of its various paths that split and merge back together) so that when the IMF package is created, the essential metadata is sitting right there in the ACESclip file.
Presently, ACESclip isn’t quite at the level of adoption for this to happen, so manual metadata setting is required at the time of IMF package creation (depending on the software tool that does this). Of course, we want to see ACESclip implemented throughout the imaging pipeline and that’s why it’s part of the ACES Logo qualifications.
Heya Andy — thanks for the response! I was worried the question was a bit too open-ended 
Appreciate the clarification! This makes a whole lot of sense, ACESclip generation happening on set… permits one to imbue CDLs created on set with meaning! Seems obvious now that I think about it.
Man, I would do anything to live in world where tracking down what the hell a given plate “is” isn’t such a monumental effort in a mixed vendor, mixed camera pipeline… fingers crossed!
Better than crossing fingers - use them to type out emails/tweets/user forum postings requesting ACESclip support from equipment manufacturers and service providers! 
Thomas: “why one of the [OpenEXR lossless] schemes was not adopted”.
As far as the compression schemes in OpenEXR being used, we took an agnostic approach here
It is an operational decision and requires that you know how your OpenEXRs are being used.
But also to the point, when compressed files are sent between facilities, there is always some likelihood
of something going wrong with a chunk and the damage is lessened if uncompressed – thus the preference for it in the ACES container. A properly check-summed file using a lossless compression scheme in the EXR library can be a good thing in some places, but adding compression/decompression blocks would slow things down a little. But the equation of where is the pain is constantly changing, so evaluating the trade of a bigger file versus better processing is always worthwhile. Also, worth mentioning, is a desire to have
the ACES clips be “Play Anywhere” and the compressed versions didn’t “Play Well” except on Big Iron. (Assuming you have the bandwidth on the machines I/O.) The compressed formats still have issues especially in 4K playback (which is where b44 came from – as a lossy fix to that problem). The ACES committee always had thought that OpenEXR had useful ‘other’ parts, and were always careful to never imply that they couldn’t be used. But by the time you are getting to interchange and archive, the benefits of uncompressed files seemed to be in the lead. As always though, changing conditions could mean that it is time to re-evaluate this question. What does everyone think?
My feeling is that EXR files are super hard to manage is smaller studios and smaller budget features. Most of our clients prefer to get masters in Pro-Res. So it’s hard to sell them an openEXR archive. I love ACES but I sometimes feel that you only have the “top tier” of productions in mind.
Thanks!
It could be an unconscious bias.
Certainly, if you start to consider TV, almost no one works uncompressed.
If you start to look at different levels of production from one-man shooter, up to 50+ camera shows, there are going to be different ‘best practices’.
I also agree about getting smaller files to deal with and this really be looked at for ACESNext.
Part of the fix may be a need for a more modern compression format in OpenEXR when you look at things like ProRes, H265, DNX*, etc The project committee didn’t have expertise in that area. Would be nice to have some.
Jim