VFX work in ACEScg with out of gamut devices

[martin.smekal]

Hello, first time in the aces discussion

What is the official ACES Team suggestion to work on footage, that has color values outside of ACEScg gamut?

I know that blue neon colors in Alexa is now a “well” known issue. But lately I came across more footage with similar issues but in other devices and light spectrums. Specifically red lights in RED cameras and red and blue lights in Blackmagic 6k pocket camera.

Both RED WideGamut and BMD Film have gamuts outside of AP0, let alone AP1 and the IDTs are not enough to accommodate for these extremes.

I work in advertising vfx and the neon / laser / LED lights are quite common in our line of work.

I know about the official blue light fix, but probably there should also be one for the red channel? Or maybe there should be some kind of tone mappings for specific cameras as well as IDTs?

Thank you very much for any suggestions.
Cheers.
Martin

[stobenkin (Steve Tobenkin)]

Thanks for your first post…welcome!

[Thomas_Mansencal]

Welcome Martin,

This is a great question, AFAIK, there is no all-in-one solution for that. It has however (and obviously) been discussed quite a few times with things along the lines of a dedicated Camera Gamut and/or Gamut Mapping.

What would be fantastic though would be to share some offending frames with the Academy: while it is easy to simulate offending cases, nothing beats having real data, and the solution model will only be as good as the input data it is fitted to.

I know that @sdyer would be pleased to get some images

Bests,

Thomas

[martin.smekal]

Thanks for the welcome!

ok, here we go
Here you can find source frames and preview jpgs, that I’m also sending here.

https://drive.google.com/drive/folders/1ZQ6fnwYOHsCcsrstOEumR_uEgfpWyata?usp=sharing

1. I have an Alexa - which is classic “Highlight Fix problem”. Blue neon sign, blue / purple stage lights, etc.

• ACES ODT Rec709
• ACES Highlight Fix Rec709
• official Alexa Rec709 lut

2. I have RED Helium and red christmas lights - lovely combination

• ACES ODT Rec709
• official RED IPP2 tonemap High Contrast Very Soft Rec709

3. I have Blackmagic Design Pocket 6k - red and blue lights, more problems visible in the red light

• ACEScct - you can see the color clamping better in ACEScct here, than in Rec709
• original BMD Pocket 6k Film Gen4 image
  
  
  
  

I think we have to do some kind of tonemapping for each color channel extreme. It would be great to have something standardized from the ACES team.

For example for the RED Log310 I know there is a conversion lut with custom tone mapping to LogC from True Color for 100 bucks…

https://truecolor.us/downloads/red-to-alexa-lut-package/

So probably something like that for multiple scenarios?

Or what would be the suggestion for these kinds of shots? For example for the RED footage with the red lights? How can I work on that shot and do VFX in ACEScg?

Thank you very much!

[Thomas_Mansencal]

@martin.smekal

Or what would be the suggestion for these kinds of shots? For example for the RED footage with the red lights? How can I work on that shot and do VFX in ACEScg?

Beside desaturating the offending hues to bring them into ACEScg gamut, there is not much you can do atm. Systematic solutions will be/are in the work but as I was mentioning it, it requires data and research

[Thomas_Mansencal]

I loaded the Red Christmas Lights image on my box quickly and I get out of the box arguably less offending results:

I exported the R3D image from RED CINE-X PRO as Ungraded RWG EXR, loaded it in Nuke with the Input - RED - Linear REDWideGamutRGB IDT. This is the upper-right image, bottom right is with a simple 20% red desaturation and ensuring the values are mostly positive:

For reference, those are the chromaticities of the image:

Cheers,

Thomas

[martin.smekal]

@Thomas_Mansencal

@martin.smekal

Or what would be the suggestion for these kinds of shots? For example for the RED footage with the red lights? How can I work on that shot and do VFX in ACEScg?

Beside desaturating the offending hues to bring them into ACEScg gamut, there is not much you can do atm. Systematic solutions will be/are in the work but as I was mentioning it, it requires data and research

That’s good to hear. I understand the real world data for research are essential. I can send more footage or rather stills with similar light situations.

[martin.smekal]

I checked my initial image with the ACES ODT and re-did it + your way in Nuke and you are totally right. The initial ACES ODT is not so severe from the get go. I don’t know why my image looks like that - I’m doing it now again and I get the same result as you…weird. Maybe my Flame + Macbook Pro GPU had hickups with the 8k R3D I’m really sorry for the misleading image.

Anyway the desaturation is good for some cases, but compared to the RED IPP2 look, it is not so solid (obviously)
Check the highlight on the knuckles for example. The edges on them.

When I show this to a colorist, I know, which way he or she will choose…

But the desaturation is probably a good start for building some gamut mapping luts maybe.

Thank you also for the graph for visual reference. Is that generated by some commercial tool or your custom one?

I’m wondering with so many vfx studios around the world using ACEScg as their working colorspace, how they go about this in a large amount of data / shots being prepared / transcoded for the artists. Is it shot by shot basis? The TDs need to “catch” these cases when the data enter the house?

@KevinJW kindly answered some questions regarding the blue channel in Alexa cameras, but because I came across these other beauties I started this topic.

Cheers
Martin

[Thomas_Mansencal]

@martin.smekal

I’m doing it now again and I get the same result as you…weird. Maybe my Flame + Macbook Pro GPU had hickups with the 8k R3D I’m really sorry for the misleading image.

No worries, it is not misleading because even with a proper conversion the problem you were underlining still exist.

@martin.smekal

but compared to the RED IPP2 look, it is not so solid (obviously)

Yes, although here we are wandering into a highly subjective territory and each camera almost needs a custom fix. The diagram highlights quite well the issue with a large chunk of chromaticities outside the spectral locus. They need to be gracefully brought back within it. You could do that with a 3x3 matrix like that @sdyer built for the Blue Light Artefact Fix LMT , but I reckon there is a need for a long term solution.

@martin.smekal

Thank you for the graph for visual reference. Is that generated by a commercial tool or your custom one?

The figure is generated with Colour, here is a How-To to produce similar images:
https://colab.research.google.com/drive/1NRcdXSCshivkwoU2nieCvC3y14fx1X4X#scrollTo=6PlESt8vGeeT

@martin.smekal

I’m wondering with so many vfx studios around the world using ACEScg as their working colorspace, how they go about this in a large amount of data / shots being prepared / transcoded for the artists. Is it shot by shot basis? The TDs need to “catch” these cases when the data enter the house?

Good question, in VFX we integrate the CG onto the plate as if it was shot by the camera, if the yellow on the knuckles is a feature, we would need to match it. Here, it would be hard with ACEScg as a large chunk of chromaticities from the camera is outside its gamut. I don’t know if there are standard practices for that as you cannot generate matching values with ACEScg in a renderer. I think I would handle those special cases in comp with some sort of gamut expansion. Keen to hear others opinion on that topic.

Cheers,

Thomas

[Troy_James_Sobotka (troy_s)]

@Thomas_Mansencal

Here, it would be hard with ACEScg as a large chunk of chromaticities from the camera is outside its gamut.

Isn’t it a bit tricky to suggest that something is outside of something else’s gamut here given everything is bound by the spectral locus? That is, what we are seeing is a direct byproduct of unfortunate math along the image pipeline?

Specifically, the fitted solve that we use for the camera is already an artificial construction as we can see from your excellent image. So the values we end up with in RGB via XYZ are already non data, or at least data that should already be gamut mapped to exist within the spectral locus. The mathematical fit yielded those artificial values.

There’d be an inherent posterization of the solved camera “primaries” as it is clipped to the spectral locus, then clipped again to the working space primaries, then clipped again to the destination output.

A feasible solution might be to gamut map along each set of camera “primaries” to the spectral locus using a nonlinear fitting?

[Troy_James_Sobotka (troy_s)]

Necromancer here, resurrecting this thread.

@martin.smekal Your images are extremely useful. Do you think the XMas lights would be available for download? I didn’t see it in your original Drive link.

Thanks so much.

[martin.smekal]

hi @Troy_James_Sobotka, you can find it still in the original drive link - in the source folder - it is the first file - it is in the original R3D format. Develop it as you wish. Thanks a lot for the interest in the topic!

A002_C029_06025K.0000000F.R3D

I saw a forum thread with a place to upload test images for WGM research, but can’t find it now…I will get back into it. I have some more real world examples to share. Just have to find the time to sort them out and upload.

[sdyer (Scott Dyer)]

@martin.smekal

I saw a forum thread with a place to upload test images for WGM research, but can’t find it now…

This one?

Submitting and Accessing Sample Images for the Gamut Mapping VWG

I have created a file request link that anyone should be able to use to upload any example images that they think would benefit the group: Submit files here You’ll need to follow a separate link in order to view the directory (i.e. to browse and download files that have been shared): View File Submissions Remember this forum is public so do be careful not to share images that you do not have rights to share. However, we do appreciate any submissions that you think would be useful and are able …

[martin.smekal]

ok, I’ve uploaded couple of beauties. Hopefully they will be helpful. Cheers!