ACES IDT DCTL Generator

@tommyzenth Cool! Thank you so much. :slight_smile: Did I miss the Linear option in the second dropdown? I couldn’t see it? I refreshed a few times.

EDIT Ah, I see the problem. It is there, but the bottom of the list is getting visually cut off by the css. Just needs a tweak to padding?

PS Is it possible for the app to take a dng sample as well as dpx? I can’t get a ‘clean’ dpx from the source dng without baking in some transform – unless someone knows how!?

Thank you very much and look forward to your news

@Dean I’ve found the problem and it is very interesting.

For Arri, the reason is that if I use the provided ACES primaries and white point ([0.73470,0.26530,0.000,1.000,0.000,-0.07700,0.32168,0.33767,0.0]) to calculate ACES(AP0) to XYZ matrix, the precision of the matrix is low compared to the matrix presented in the ACES P-2013-001document. As a result, I directly use the XYZ matrix shown in ACES P-2013-001document with the CAT02 method, I got the exact same transformation (even higher precision) as the Arri official document. (I have updated the tool)

@sdyer I think a more accurate ACES RGB primaries should be provided to make all the calculations consistent.

As for RED, there is a very tiny deviation at ~0.00002 precision caused by the Bradford matrix, as I only got 4digit precision data of the Bradford matrix. The deviation will be fixed if I find a Bradford matrix model with higher precision.

@sdyer I also found a very interesting point about the white point D65, the CIE xy data of D65 is
[0.3127, 0.3290] and its XYZ value is shown as [0.9504, 1.0000, 1.0888] on every document I found. However, when D65 xyY is converted to XYZ value mathematically, it can never be [0.9504, 1.0000, 1.08883]. It is not a precision issue. I am not sure what’s going on here.

Hi,

The primaries are what they are and they should not be changed, ever :slight_smile:
Likewise, the Bradford matrix is given at 4 digits and that is what you will find everywhere, should you change it for something more precise, and you would never match other sources.

The CIE xy values for D65 is not [0.3127, 0.3290] but [0.31272, 0.32903] as given in CIE 015:2018 Colorimetry, 4th Edition. The rounded value at 4 digits is the one used by virtually all the RGB colourspaces using D65. You will, unfortunately, find many variants depending on the source of the data.

>>> import colour
>>> colour.xy_to_XYZ(colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'])
array([ 0.95045593,  1.        ,  1.08905775])
>>> colour.xy_to_XYZ([0.31272, 0.32903])
array([ 0.95043005,  1.        ,  1.08880649])
>>> colour.sd_to_XYZ(colour.ILLUMINANTS_SDS['D65']) / colour.sd_to_XYZ(colour.ILLUMINANTS_SDS['D65'])[1]
array([ 0.95046506,  1.        ,  1.08897024])
>>> colour.sd_to_XYZ(colour.ILLUMINANTS_SDS['D65'], method='Integration') / colour.sd_to_XYZ(colour.ILLUMINANTS_SDS['D65'], method='Integration')[1]
array([ 0.95046532,  1.        ,  1.08897069])
>>> colour.HUNTERLAB_ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'].XYZ_n / colour.HUNTERLAB_ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'].XYZ_n[1]
array([ 0.9502,  1.    ,  1.0882])

The important thing is to be aware of it and accept the fact that there will be precision issues.

Cheers,

Thomas

@tommyzenth
Update on visual display of Linear option et al. Just a heads up: I’m on Mac: Linear option is only visible on Firefox. Safari and Chrome it is cut-off. First tests of the XYZ > Linear > ACES settings create a DCTL that is visually identical to leaving Resolve CTL to ‘No LUT selected’. I assume this means that Resolve is using the Adobe DNG D50 matrices? Thanks again!

Got it. Thanks a lot for the insight! The explanation is quite informative, I really appreciate it.

Thanks for the feedback!. I will check if there is a bug for Safari or Chrome soon. I found the visual result identical to leaving Resolve No Lut is caused by a mistaking when implementing CIE XYZ space as source space. I really hope if you could send me one ML sample dng file. That would be sufficient to test because currently it quite hard for me to debug ML raw without any visual content.

@tommyzenth Great! I will email you a ML dng now. :slight_smile:

@rivertim I currently got correct procedure. I will email you!

@Paul_Dore Sorry if this is a dumb question, but is it impossible to intercept the transform in Resolve from Raw XYZ into ACES that it performs automatically when say DNG is loaded?

Hi,

I just fixed a bug that the previous display in rec709(ODT+RRT) may encounter incorrect white point issue due to an additional matrix transpose operation, now it’s fixed. (The IDT were not affected, only the display in web page had a bug).

@rivertim @Dean Just let you guys know.

@tommyzenth Thanks for the fix!

@tommyzenth Thanks :+1:

The formula converting XYZ to ACES is universal, what you want to look at is the conversion from the camera raw to XYZ. There’s no need for a transfer function to convert to linear (because it’s already linear in raw), just two 3x3 matrices (one for the conversion to XYZ and one for the white point conversion). As far as I know, you can’t intercept these values directly in Resolve. You would have to overwrite the DNG metadata if you want to change the values. Otherwise, you could apply an additional 3x3 matrix via DCTL (provided you know which values are being interpreted in the metadata).

Another issue with the DNG format is that it’s not really suited for true scene referred caption i.e. the full dynamic range of the camera is always squeezed into the region of 0.0 to 1.0, so even in linear space there’s no extra highlight latitude (when there should be). This was one of the reasons I worked on a DNG specific DCTL.

@Paul_Dore Thanks for the reply Paul. This is very useful information. Is this ‘two 3x3 matrices’ data that could theoretically be re-written to the DNG post capture using say exiftool? Fascinating re the extra highlight latitude missing in DNG format. I didn’t know this. Does ensuring the DNG has max 16383 White Level present improve headroom? Or is it data we can never recover?

Hi,

I added four more Gamuts according to the recent update of ACES 1.2.

  • Sony VENICE S-Gamut3
  • Sony VENICE S-Gamut3.Cine
  • Sony S-Gamut Daylight
  • Sony S-Gamut Tungsten

hi tommy, thanks for your work,
is there any chance to include Blackmagic cameras to your list?
cheers

@osama_rasheed I added 6 BMD colorspace as follows :slight_smile:

  • BMD 4.6k Film Gen 1
  • BMD 4.6k Film Gen 3
  • BMD 4k Film Gen 1
  • BMD 4k Film Gen 3
  • BMD WIde Gamut Film Gen 4 BMPCC 4k Film Gen 4
  • BMD Video Gen 4

The log curve data of the BMD cameras are not published, currentlly you may need to convert the log gamma to other common gamma (linear, arri log C, etc
) via Color Transform Ofx or 1D Lut in Resolve before using this DCTL. :grinning:

Wow great for the help. Thanks for that

Nice! Are the BMD matrices reverse engineered out of the Resolve Color Space Transform OFX?

I’ve not experimented with it, but DCTLs can apply LUTs, so I wonder if it might be possible to have generated DCTLs for BMD transforms make use of the 1D LUTs which ship with Resolve to do the linearisation?

EDIT
Just tested, and as an example the following DCTL seems to work:

DEFINE_LUT(BMDFilm46, ../VFX IO/BMDFilm 4.6K to Linear.cube)

__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
  const float mtx[9] = {0.6472787323041167f,0.2410444287328754f,0.11167683903633815f, 0.06552081891597149f,1.0229602491189127f,-0.08848106807427779f, -0.027417492153046195f,-0.07610880957393346f,1.1035263016789891f};
  
  float3 rgb = APPLY_LUT(p_R, p_G, p_B, BMDFilm46);
  
  float r2 = rgb.x * mtx[0] + rgb.y * mtx[1] + rgb.z * mtx[2];
  float g2 = rgb.x * mtx[3] + rgb.y * mtx[4] + rgb.z * mtx[5];
  float b2 = rgb.x * mtx[6] + rgb.y * mtx[7] + rgb.z * mtx[8];

  return make_float3(r2, g2, b2);
}

Note: The relative path of the LUT needs to be correct. I have a first level sub-folder of IDTs, hence the ../ at the start of the LUT path. I only tested on a Mac. Cross platform compatibility might complicate path naming.