Proposed AMF spec update to support linking custom transforms

Discussions - ACESNext VWG – ACES Metadata File (AMF)
1

It has been brought up in multiple AMF implementation meetings that in order to support linking to non-ACES transforms, which is common for unpublished IDTs and user-generated looks as LMTs, the current spec of only using <transformID> is insufficient.

ACES Transform ID’s (<transformID>) are important for core ACES transforms and version control, but for anything outside of that it seems to be unclear how this ID is intended to be used.

For example, if a transform is generated and assigned a custom transform ID (example: IDT.Acme.NewCameraX.a1.v2 or LMT.ShownameX.bleachbypass.a1.v2) how will another product know what that means and where to find it? Would it just be the filename of the transform (with or without file extension)?

Examples from implementers have been that in these scenarios, it might be easiest to use filename - since most software has a folder where all the LUTs live - and maybe more robust to use a filename + UUID.

The following proposal would be a small addition to the current spec released with ACES v1.2.

The following would be added under <inputTransform> and <lookTransform>.

<inputTransform>

  • <file> - this can be duplicated from <clipID>
  • <uuid> - this can be duplicated from where UUID already exists

<lookTransform>

  • <file> - this can be duplicated from <clipID>
  • <uuid> already exists here

While we are updating the spec, should <transformID> be changed to <ACEStransformID> in order to be in line with the CLF element? This would also make it clear that this refers to the ACES Transform ID as specified in S-2014-002 and not some other generic ID.

Here is how this might look with a CLF as the external transform (thank you @Giardiello and @walter.arrighetti for helping with this proposal):

image1
image11999×1810 128 KB

How does everyone feel about this? We think it would be a useful addition to the AMF spec which should help implementation and for people to use custom transforms.

Thanks,
Chris

2

Hi Chris,

About your last proposal for the name of the TransformID tag, I agree in principle, but an ACES’ XML namespace is in place in AMF, therefore the real tag is already <aces:transformID>.

Instead than modifying this in AMF, I suggest to allow ACES XML namespace in CLF (at least optionally), which harmonizes CLF and AMF name tags, currently existing with different names for similar or identical purposes (like the acesTransformId / TransformID discrepancy that you correctly spotted).

3

Hi Chris,
regarding the core proposal of the thread now.

I’d like to add that the same method for linking external transforms in IDT and LMTalso holds for ODT, therefore the proposal that we designed is extensible, as is, to ODT.

I think it is useful to extend to the group at large the presentation deck
Advanced workflows interoperating AMF, CLF and CDL, by myself and @Giardiello.

It has are some workflow diagrams and, above all, code examples showing the details of all alternatives for linking AMF with CLF and CDL, line by line.

In fact, with the same strategy, we get for free the automatic linking of external CDL from the LMT, thus opening up other “advanced workflows” that we designed exploiting AMF’s current spec (with only minor addition proposal).

The advanced workdlows are shown in the same slides’ deck above.

4

Hi Chris,

I think your proposal is excellent. I agree that establishing the linkage from the AMF to an external file is most pragmatically done via a filename.

I also agree that we should change the AMF <transformID> to <ACEStransformID>. It clarifies the semantics of that element and also better aligns with CLF.

Regarding the use of a hash with CLF, there is currently no specification for how that should be calculated and there is no hash value in the CLF itself.

It would be helpful to implementers to provide some more detail on how the <uuid> element should be used. For example, is this just used as a check after the file has been located?

Some additional clarification might help regarding <inputTransform> and how <file> is used when the <inverse...> elements are used. Is it correct to say that the <file> is not allowed, just like it is not allowed with the forward Output Transforms?

Regarding one of Walter’s suggestions, the decision to allow <file> with Output Transforms would seem to have major architectural implications since it potentially would open up AMF to be used with non-ACES pipelines.

Doug

5

Hi Doug,
I will try to reply to your comments one by one:

1.
Since XML namespaces are used in AMF (this is standard practice in contemporary XML, as suggested by W3C), changing the name of transformID tag into ACEStransformID has a side effect that, when the full namespace is accounted for, one ends up having tags like:
<aces:ACEStransformID>
which makes no sense, because of the repetition and because the Transform ID isn’t a “more of an ACES tag” than other tags are.
In my opinion, it makes sense to harmonize AMF and CLF further, by using mandatory namespace in CLF as well (and renaming CLF’s tag ACEStransformID back to AMF’s transformId).

2.
Regarding the UUID, the presentation (slides #8 through #13) by myself and @Giardiello, that circulated in ACEScentral and Dropbox, graphically clarify the usecases, and detail out which part of CLF links to which tags in AMF and vice versa. Please read them and reach out to me if you need further clarification.

3.
Hash-linking was proposed (and already is in the AMF specs for the LMT component) as an optional linking strategy. That is not at all intended to be the ideal way to do that – not even the simplest one. In certain situations thogh --where one wants to be absolutely sure no “amateur” screws up the color pipeline pre-determined by the Director of Photography and a color-science department, elsewhere at a DI facility-- that might be a good tactics to go to.

Please also note that, in case of hash-linking, there is nothing else to do at CLF level: the application creating the AMF file will also compute the hash of the CLF file and put in the AMF’s <hash> tag. This can be accomplished by one line of code in most programming and scripting languages. All the existing DCP- or IMF-aware applications already do this, as hash is the one and the industry-standard way of cross-linking files in DCP and IMF.
We re-use what someone else has already found to be effective, and has been successfully using ever since.

Hash computation is standardized by W3C and there are many several possible algorithms. The most used one today is SHA-256, because SHA-1 and MD5 have been proven to be weak. Yet, this is not something to be concerned about since --again-- hash-computation is 1-line of code, no developer need to be a cryptography expert to use it.
I agree with you: it’s best to enter a choice of 1 or 2 algorithms in the AMF specs, so that all implementations agree with them.

4.
In my opinion if external files are linkable, every option --even non-ACES workflows-- may be possible: you just need to use as IDT a CLF that has a non-ACES destination colorspace (or vice versa) and you’re done. No metadata will prevent you to practically realize that.
Furthermre, why should we deny someone with a LUT realizing the role of an inverse ODT to use that as an IDT (and vice versa) ?
Finally, I agree that using file tag for ODTs may potentially open up to non-ACES workflows but --again-- not more easily than one can exit ACES by referring external IDTs or external LMTs. I don’t think this is going to be any riskier because we allow that with ODTs.
It could be riskier with externally-referenced RRTs: that is why we’d like to prevent file-linked RRTs.