VFX work in ACEScg with out of gamut devices

Hello, first time in the aces discussion :wave:

What is the official ACES Team suggestion to work on footage, that has color values outside of ACEScg gamut?

I know that blue neon colors in Alexa is now a “well” known issue. But lately I came across more footage with similar issues but in other devices and light spectrums. Specifically red lights in RED cameras and red and blue lights in Blackmagic 6k pocket camera.

Both RED WideGamut and BMD Film have gamuts outside of AP0, let alone AP1 and the IDTs are not enough to accommodate for these extremes.

I work in advertising vfx and the neon / laser / LED lights are quite common in our line of work.

I know about the official blue light fix, but probably there should also be one for the red channel? Or maybe there should be some kind of tone mappings for specific cameras as well as IDTs?

Thank you very much for any suggestions.
Cheers.
Martin

2 Likes

Thanks for your first post…welcome!

Welcome Martin,

This is a great question, AFAIK, there is no all-in-one solution for that. It has however (and obviously) been discussed quite a few times with things along the lines of a dedicated Camera Gamut and/or Gamut Mapping.

What would be fantastic though would be to share some offending frames with the Academy: while it is easy to simulate offending cases, nothing beats having real data, and the solution model will only be as good as the input data it is fitted to.

I know that @sdyer would be pleased to get some images :slight_smile:

Bests,

Thomas

Thanks for the welcome!

ok, here we go
Here you can find source frames and preview jpgs, that I’m also sending here.

https://drive.google.com/drive/folders/1ZQ6fnwYOHsCcsrstOEumR_uEgfpWyata?usp=sharing

  1. I have an Alexa - which is classic “Highlight Fix problem”. Blue neon sign, blue / purple stage lights, etc.
  • ACES ODT Rec709
  • ACES Highlight Fix Rec709
  • official Alexa Rec709 lut

  1. I have RED Helium and red christmas lights - lovely combination :slight_smile:
  • ACES ODT Rec709
  • official RED IPP2 tonemap High Contrast Very Soft Rec709

  1. I have Blackmagic Design Pocket 6k - red and blue lights, more problems visible in the red light

I think we have to do some kind of tonemapping for each color channel extreme. It would be great to have something standardized from the ACES team.

For example for the RED Log310 I know there is a conversion lut with custom tone mapping to LogC from True Color for 100 bucks…

https://truecolor.us/downloads/red-to-alexa-lut-package/

So probably something like that for multiple scenarios?

Or what would be the suggestion for these kinds of shots? For example for the RED footage with the red lights? How can I work on that shot and do VFX in ACEScg?

Thank you very much!

Beside desaturating the offending hues to bring them into ACEScg gamut, there is not much you can do atm. Systematic solutions will be/are in the work but as I was mentioning it, it requires data and research :slight_smile:

I loaded the Red Christmas Lights image on my box quickly and I get out of the box arguably less offending results:

I exported the R3D image from RED CINE-X PRO as Ungraded RWG EXR, loaded it in Nuke with the Input - RED - Linear REDWideGamutRGB IDT. This is the upper-right image, bottom right is with a simple 20% red desaturation and ensuring the values are mostly positive:

For reference, those are the chromaticities of the image:

Cheers,

Thomas

1 Like

That’s good to hear. I understand the real world data for research are essential. I can send more footage or rather stills with similar light situations.

I checked my initial image with the ACES ODT and re-did it + your way in Nuke and you are totally right. The initial ACES ODT is not so severe from the get go. I don’t know why my image looks like that - I’m doing it now again and I get the same result as you…weird. Maybe my Flame + Macbook Pro GPU had hickups with the 8k R3D :slight_smile: I’m really sorry for the misleading image.

Anyway the desaturation is good for some cases, but compared to the RED IPP2 look, it is not so solid (obviously)
Check the highlight on the knuckles for example. The edges on them.

When I show this to a colorist, I know, which way he or she will choose… :slight_smile:

But the desaturation is probably a good start for building some gamut mapping luts maybe.

Thank you also for the graph for visual reference. Is that generated by some commercial tool or your custom one?

I’m wondering with so many vfx studios around the world using ACEScg as their working colorspace, how they go about this in a large amount of data / shots being prepared / transcoded for the artists. Is it shot by shot basis? The TDs need to “catch” these cases when the data enter the house?

@KevinJW kindly answered some questions regarding the blue channel in Alexa cameras, but because I came across these other beauties :slight_smile: I started this topic.

Cheers
Martin

No worries, it is not misleading because even with a proper conversion the problem you were underlining still exist.

Yes, although here we are wandering into a highly subjective territory and each camera almost needs a custom fix. The diagram highlights quite well the issue with a large chunk of chromaticities outside the spectral locus. They need to be gracefully brought back within it. You could do that with a 3x3 matrix like that @sdyer built for the Blue Light Artefact Fix LMT , but I reckon there is a need for a long term solution.

The figure is generated with Colour, here is a How-To to produce similar images:
https://colab.research.google.com/drive/1NRcdXSCshivkwoU2nieCvC3y14fx1X4X#scrollTo=6PlESt8vGeeT

Good question, in VFX we integrate the CG onto the plate as if it was shot by the camera, if the yellow on the knuckles is a feature, we would need to match it. Here, it would be hard with ACEScg as a large chunk of chromaticities from the camera is outside its gamut. I don’t know if there are standard practices for that as you cannot generate matching values with ACEScg in a renderer. I think I would handle those special cases in comp with some sort of gamut expansion. Keen to hear others opinion on that topic.

Cheers,

Thomas

1 Like

Isn’t it a bit tricky to suggest that something is outside of something else’s gamut here given everything is bound by the spectral locus? That is, what we are seeing is a direct byproduct of unfortunate math along the image pipeline?

Specifically, the fitted solve that we use for the camera is already an artificial construction as we can see from your excellent image. So the values we end up with in RGB via XYZ are already non data, or at least data that should already be gamut mapped to exist within the spectral locus. The mathematical fit yielded those artificial values.

There’d be an inherent posterization of the solved camera “primaries” as it is clipped to the spectral locus, then clipped again to the working space primaries, then clipped again to the destination output.

A feasible solution might be to gamut map along each set of camera “primaries” to the spectral locus using a nonlinear fitting?

1 Like

Necromancer here, resurrecting this thread.

@martin.smekal Your images are extremely useful. Do you think the XMas lights would be available for download? I didn’t see it in your original Drive link.

Thanks so much.

hi @Troy_James_Sobotka, you can find it still in the original drive link - in the source folder - it is the first file - it is in the original R3D format. Develop it as you wish. Thanks a lot for the interest in the topic!

I saw a forum thread with a place to upload test images for WGM research, but can’t find it now…I will get back into it. I have some more real world examples to share. Just have to find the time to sort them out and upload.

This one?

1 Like

ok, I’ve uploaded couple of beauties. Hopefully they will be helpful. Cheers! :slight_smile:

2 Likes