Proposed list of test material (AMF and reference images)

Discussions - ACESNext VWG – ACES Metadata File (AMF)
1

As discussed in the last VWG meeting (2020-03-05) you can now find the current list of existing and planned test material for AMF below. This list and the material is intended to be finished until “end of phase 1”, currently planned for end of March.

Proposed list of test material

The goal of this material is to provide vendors who implement AMF a set of test files that they can use to verify if:

  • AMF files can be read,
  • warnings and errors are displayed properly (e.g. for version mismatch),
  • the pipeline recreates the correct image.

AMF Files Only

basic test vectors (succeeding and failing)

  • for instance AMF files with version checks
  • by Dan Tatut, Marquise Technologies

syntax stress test

  • examples with varying XML syntax (non-canonical)
  • examples with namespace prefix
  • by Walter Arrighetti
  • Post on ACES Central
  • AMF files: ZIP file (retrieved 2020-Mar-06)

AMF Files with Reference Images

The current set of material from this section can be found here:
https://github.com/pomfort/amf-util (see continuously updated download link in the README).

Source images:

  • Original camera sources (LogC and S-Log3) have been provided by Joseph McCormick and Chris Clark (Netflix)
  • Original camera sources provided by the Academy
  • DCDM examples are provided by by Dan Tatut, Marquise Technologies (TBD)
  • sRGB example created by Patrick Renner, Pomfort
  • Rec.709 example created from original camera sources by Patrick Renner, Pomfort

Alexa LogC IDT (without CDL, LMTs)

  • ODTs: P3 D65, HDR PQ, Rec.709, P3 D65 HDR
  • LogCEI800-P3D65-D60sim48nits.amf
  • LogCEI800-Rec.2020ST20841000nits.amf
  • LogCEI800-Rec.709100nitsdim.amf

Sony S-Log3 IDT (without CDL, LMTs)

  • ODTs: HDR PQ, Rec.709
  • S-Log3S-Gamut3-Rec.2020ST20841000nits.amf
  • S-Log3S-Gamut3-Rec.709100nitsdim.amf

Rec.709 “IDT” (without CDL, LMTs)

  • invODT+invRRT
  • ODT: Rec.709
  • Rec.709-Rec.709100nitsdim.amf

sRGB “IDT” (without CDL, LMTs)

  • invODT+invRRT
  • ODT: P3 D65 48nits
  • sRGB-P3D6548nits.amf

OpenEXR in ACES AP0 for VFX (without CDL, LMTs)

  • skipping IDT
  • ODT: Rec.709
  • LogCEI800-Rec.709100nitsdim__skip-IDT.amf

DCDM

  • XYZ TIFF input
  • invODT+invRRT
  • ODT: Rec.709
  • XYZ-Rec.709100nitsdim.amf

TBD: camera source with CDL in several WCS

  • e.g. Alexa LogC IDT
  • WCS
    • ACEScct + ASC-CDL
    • LogC_EI800_AWG
    • ADX10

camera source with LMT via transformId

  • blue highlight fix LMT
  • ArriAlexa.LowLight.0117185 (shows blue highlights)
  • reference images both with and without LMT applied
  • ODT: Rec.709

TBD: camera source with LMT via CLF

  • not clear yet how to create reference images
2

Hi Patrick.
This is really good material, thanks for sharing this and sorry for not providing feedback earlier.
I’d love to prepare other XML stress test AMFs to add.

As for the picture, during the last call I proposed to include a few test charts / slates with at least the following:

  • black/white gradient
  • 3 complementary-color gradients (suggested bu @joachim.zell)

I was told the gamut-mapping VWG is working on such test charts for other reasons, so this can be borrowed. The reason for the gradients is to test whether a multi-layer AMF is processed in the right order (e.g. ACEScct conversion happens before CDL before applying a LUT, before the RRT, etc.): if any ordering fails, the gradients are easier to detect problems.

Finally, it would be good if each of the original camera images is processed through all the output pipelines: currently, for example, the S-Log3S sources are only processed into Rec.2020 (ST2084) and Rec.709 (100nit).

3

Hello Walter, thanks for your feedback.

I can add the charts with grey- and color-ramps easily when they are available. Are you part of that VWG and can you make us aware when these are available?

About the combinations of output transforms: My rationale was to provide at least enough combinations that setting up the ACES pipeline can be tested for now –but not the entirety of the pipeline transforms themselves.

We could extend the test material to a new, full set of combinations (IDTs vs. ODTs). That could then become a test set for evaluating the correctness for an entire ACES implementation. But to me that’s currently beyond the scope of this group.

But I totally agree that such a test set would be great to have for the ACES community – and that it can be easily created with our current tooling. If you think we need to merge these efforts into one large test set, I think we should then also extend the range of source images to more cameras, so that we don’t end up processing the same logC image with all kind of IDTs (which would reference result in weird result images sometimes).