Proper Texturing Workflow for Unreal Engine

Discussions - Using ACES Video Games
1

Hi folks,

I’m a graphics engineer learning about ACES for the first time. I’ve started working on an Unreal project, after they moved to the ACES tonemapper.

When we started developing a look for our game, we found that the textures we were creating in Substance Painter seemed ‘washed out’ when we brought them into Unreal. Specifically, certain yellow and grey hues ended up near the same, and saturation as a whole seemed decreased. After setting up a matching HDRI scene in Unreal and a bit of digging, we found that this was due to the ACES tonemapping step; Post Tonemap HDR Color (which seems to actually skip the actual tonemap) matched the look in Substance, but the final image did not.

We then found https://share.substance3d.com/libraries/4450 to match substance to Unreal, though upon closer inspection it looks to be using the simplified fit from: https://knarkowicz.wordpress.com/2016/01/06/aces-filmic-tone-mapping-curve/ which doesn’t include some of UE4’s tweaks. It’s close enough that we were able to start getting consistent looks between Substance and Unreal. I’ve since stumbled across https://www.artstation.com/artwork/mrqd8 from @bleleux and https://www.artstation.com/artwork/2xkEbY from @Dogway.

My concern is that we may be texturing to meet UE’s default tonemapping and then find later on if we adjust it or as we color grade that we find our values are off. I’ve read through https://chrisbrejon.com/cg-cinematography/chapter-1-5-academy-color-encoding-system-aces/ and am familiar with Seb Lagarde’s albedo charts which give a good general guideline for base colour. First I have some assumptions I want to validate:

  • UE4 uses ACES 1.0.3 and not 1.1?
  • UE4 uses sRGB linear (given you’re feeding it sRGB colour) throughout the pipeline up to the tonemapping step, at which point it uses RRT -> ODT to output the final tonemapped image?
    • At this point, it seems prohibitive to move all textures to be pre-converted to ACEScg
  • UE4 has a fix for blue intensity applied to ACES similar to High light fix and premature desaturate?
  • The other tweak is a gain of ~1.4, which I believe I read was to limit the amount of texture rework and still use sRGB.
    • My understanding is that actually ends up having an effect similar to increasing exposure
    • As intensity increases, the ACES tonemapper ends up desaturating colours
  • ACES 1.1 solves some of the hue shifts issues from ACES 1.0.3 (possibly removing the blue fix in Unreal)

And now my questions:

  • Why is the ACES tonemap applied on linear sRGB instead of using an IDT to transform from sRGB -> ACEScg? Or is there an IDT applied there and I missed it?
  • From Substance to Unreal, which is the more ‘correct’ workflow?
    • No changes to materials, and using @bleleux’s ACES LUT?
      • Is it correct to say that this has baked the entire IDT -> ACES -> RRT -> ODT transform into one lookup?
    • @Dogway’s filters + LUT, where for display in Substance both the PBR_SmartFit and ACEScg conversion are applied
      • For export since we’re still using sRGB in Unreal, we would export the result from the PBR_SmartFit filter?
      • The discussion on @Dogway’s posts and @Thomas_Mansencal’s responses seems to indicate there’s disagreement over the usage of PBR_SmartFit to remap albedo colours?
    • The ACES per material shader we found (but possibly expanded to include UE4’s tweaks to ACES)?
  • When should we (or do you) adjust the tonemapping curve to ensure our textures are supporting the result we want?
    • How do you help ensure your albedo textures are ‘base truth’ and not compensating for the tonemap apart from having a physically based reference for all material types?
      • Of course the albedo charts help here
  • Are we shooting ourselves in the foot matching to UE’s implementation of ACES tonemapping with its tweaks, particularly the gain? Would we be better served by replacing it with ACES 1.1 and then using the adjustments provided by Unreal (i.e. global, highlights, shadows settings)?

And for anyone else who ends up coming across this thread and wants more reading:

1 Like
2

Correct although for LDR (non-HDR) output it does not really matter as the LDR ODT is the same and you would need to grade specifically for HDR.

Also correct, UE4 working space is implicitly assumed to be sRGB.

Yes but it is not implemented as per the book and cannot be matched in a standard ACES worfklow. There is also a Gamut Expansion LMT in UE4, if you have UDN access, this is super relevant: https://udn.unrealengine.com/questions/543111/view.html

Correct, it is exactly 1.45.

No, for practical purposes, ACES 1.1 and ACES 1.0.3 are the same for the LDR path.

IDTs are technically for input devices, e.g. a camera and they convert from that input device to ACES2065-1 (AP0). UE4, internally converts directly from sRGB to ACEScg (AP1) as required for tonemapping purposes.

Yes, however, the conversion is more like sRGB —> ACES2065-1 —> RRT —> ODT.

No please don’t do that, it is a landmine field and it will throw you off a cliff one day or the other. I wish that thread was never referenced again.

We did that at some point but it is much easier to use a LUT, even one that clips highlights. Keep in mind OCIO is coming to Substance Painter and it will make all the three methods above obsolete.

Never! :slight_smile: By adopting a fixed View/Display Rendering Transfom (DRT), you ensure that all your assets are authored in a consistent way which makes them easy to share across shows/games. The other side of the coin is that if they are authored while respecting PBR common sense, they should look good no matter the DRT.

Ideally, you have everything under control and processed the photographic source images linearly, with reference colour rendition charts, etc… Now it is important to keep in mind that “base ground truth” is somehow vague: albedo in the first place is a measure bound to solar irradiance which encompasses both the diffuse and specular components of the reflection when it is measured. Textures are pretty much never shot in direct sunlight :wink: Ground truth would be more about spectrally measure not only the BTF (Bidirectional Texture Function) of the surface but also measure its Polarimetric Reflectance: a Spatially Varying Polarimetric BRDF (Bidirectional Reflectance Distribution Function). And then there is fluorescence, etc… :slight_smile:

ACES 1.1 will not change anything and we are at 1.2 btw :slight_smile: The easiest way, for now, is to use the following OCIO colourspaces and disable the Gamut Expansion and Blue Light Artifact Fix as indicated in the UDN Question:

   - !<ColorSpace>
     name: ACES sRGB ODT
     family: View
     equalitygroup: ""
     bitdepth: 32f
     isdata: false
     allocation: uniform
     allocationvars: [0, 1]
     to_reference: !<GroupTransform>
       children:
         - !<FileTransform> {src: InvRRT.sRGB.Log2_48_nits_Shaper.spi3d, interpolation: tetrahedral}
         - !<FileTransform> {src: Log2_48_nits_Shaper_to_linear.spi1d, interpolation: linear}
         - !<ColorSpaceTransform> {src: Colourspace - ACES 2065-1, dst: Model - Reference}
     from_reference: !<GroupTransform>
       children:
         - !<ColorSpaceTransform> {src: Model - Reference, dst: Colourspace - ACES 2065-1}
         - !<FileTransform> {src: Log2_48_nits_Shaper_to_linear.spi1d, interpolation: linear, direction: inverse}
         - !<FileTransform> {src: Log2_48_nits_Shaper.RRT.sRGB.spi3d, interpolation: tetrahedral}
 
   - !<ColorSpace>
     name: UE4
     family: View
     equalitygroup: ""
     bitdepth: 32f
     isdata: false
     allocation: uniform
     allocationvars: [0, 1]
     to_reference: !<GroupTransform>
       children:
         - !<ColorSpaceTransform> {src: ACES sRGB ODT, dst: Model - Reference}
         - !<CDLTransform> {slope: [1.45, 1.45, 1.45], direction: inverse}
     from_reference: !<GroupTransform>
       children:
         - !<CDLTransform> {slope: [1.45, 1.45, 1.45]}
         - !<ColorSpaceTransform> {src: Model - Reference, dst: ACES sRGB ODT}

Should use the ACES Config with that new ColorSpace and it will match UE4.

Cheers,

Thomas

3

Hello, my disagreement with @Thomas_Mansencal was ONLY about having the option for “Output - sRGB” as IDT (talk about nitpicking). But as I said it’s an option not an enforcement, you can also choose “Utility - sRGB - Texture” as IDT to conform to standard ACES.

Use common sense, applying a gain of 1.45 because “Utility - sRGB - Texture” looks dark is a destructive hack, you will blow out highlights, think of white textures (walls, snow, etc).

I will be releasing a project in 3 weeks most that deals with all the issues people are having with “Utility - sRGB - Texture”.

Using the contained LUTs that do sRGB —> ACES2065-1 —> RRT —> ODT is a landmine field, colors don’t go well through the log shaper for Substance Painter LUTs. My version doesn’t do any color work, it’s only the tonemapping part.

Now I also have other LUTs that along the ACES Filter (PBR SmartFit is for PBR fit) makes it possible to go through ACES within Substance and later apply the RRT + ODT, and this is close to official ACES as you can check here.

You can still use PBR SmartFit for PBR compliant textures though, this ensures physical plausible albedo ranges as PBR guidelines (PBR MetalFit includes fix for metals too), you still don’t know if it has embedded camera curves but if you author your own textures you can design a negative curve of your camera response (i.e. shooting to some targets), reference colour rendition charts by their own are not reliable enough by the way.

4

This is a gain on the camera exposure values, and not the textures.

5

I know, you can use “gain/exposure” at a Substance level (along the plain contained LUT) if that’s the workflow you want to follow.

6

Looks like Thomas answered everything :grinning:

The gamut expansion is subtle enough to ignore for the average texture in my experience unless you’re working with saturated colors near the Rec.709 boundary. The blue light fix may be more of a common issue since it has a bit of an effect into cyan and purple, but the blend is arbitrary and may not even be necessary. I generally disable it unless I need it- it’ll be pretty obvious if you have any bright blue emissives and don’t use the correction.

I agree. There’s a few shader-based ACES implementations(almost always from Narkowicz’s RRT+ODT fit), but imo it helps to have the background HDRI match as well and it reduces the need to manage however many Painter shaders you use. Sadly the LUT requires an additional step that I wish wasn’t needed, but you can set up a template with everything already enabled for your project.

1 Like
7

Thank you all so much for your quick responses. Couple follow-ups:

To make sure I’m interpreting this correctly: you’re advocating avoiding touching any of the filmic curve parameters UE exposes like slope/toe/shoulder, since that is meant to be a fixed standard, and any adjustments to be done through LMTs (UE’s color grading categories)?
To go back to what spurred this topic, the original rock texture done in Substance, using their default linear tonemapper, had a lot of colour detail that ended up disappearing after tonemapping. Am I correct that this likely means our albedo was authored with too bright colours? And we should author based on no changes to the color grading categories & the default ACES curve.

I read the UDN link - to disable these changes, you locally modified the PostProcessCombineLUT shader, correct?

Thank you for clarifying Jose; I must admit that at first that discussion had gone over my head, so the exact disagreement was unclear.

What is the ‘log shaper’? Which part of the transform does it affect?

Good to know. I’m going to see what effect it has on our current content if we revert back to standard ACES, but depending on our VFX going forward I may need to bring back the blue light fix as you point out.

8

Exactly! It is entirely fine to play with exposure though as you author your assets, you should do that all the time in fact, check that they look great at any exposure value.

It could be either that the albedo values are too high or that the lighting intensity/exposure of the scene is too high. Hard to get a solid answer without seeing the texture or asset.

Ideally yes, see it as your neutral authoring conditions.

You can actually set them to 0 in the PostProcessVolume Actor:

image

Cheers,

Thomas

9

Thanks for this great thread ! There were a couple of similar topics on the forum in case you haven’t read them :

Mostly answered by @Thomas_Mansencal :wink: There is probably more stuff to read about albedos and such but I would have to dig deeper in the forum archives.
We are also trying to develop an open-source tool based on the Pointer’s Gamut to help artists to check their textures’ base colors (as a personal initiative, not ACES related). Maybe it could be helpful.

Regards,
Chris

10

It is certainly worth collecting all that in a document that can be linked directly, given how many times it came, it makes sense to do that.

Cheers,

Thomas

1 Like
11

There was a discussion about a knowledge base on the new ACESCentral website. Like collecting all the interesting threads into proper documents/pages to give people easier access. I would even volunteer to do that to be honest, since my website is partially based on these threads. Will ask Steve and Scott about it. Sorry, I did not mean to hack the thread… :wink:
Chris

12

No worries @ChrisBrejon! I’m sure a lot of folks would benefit from that page; your summary that I linked earlier was far and away one of the best introductions to this topic.

I’m also interested in the tool you mentioned; I’d proposed building something similar to my team, along the lines of https://docs.unity3d.com/2018.1/Documentation/Manual/MaterialValidator.html

13

Cool ! I’ll let you know how this goes @daskyler Thanks for the link to the Unity page. I didn’t know about it ! Sebastien Lagarde and his team really care about this topic : PBR values and such… :wink:

14

I was getting there and would have volunteered you :smile:

15

At BioWare after we’d transitioned to Frostbite, Seb Lagarde & Charles de Rousiers helped guide our team through the transition to PBR. Can’t speak highly enough of them :slight_smile:

16

Several of the sections in the ACES Central Knowledge Base are summarised versions of threads or posts from the forum. I think it would be worth talking to @stobenkin about adding other topics to that, if you have ideas of what would be useful. It could benefit from more content there, and the useful forum discussions can be hard to keep track of.

1 Like
17

FYI Actually @sdyer has been maintaining the knowledge base.

18

and we’re working on it! :slight_smile:

19

What is the ‘log shaper’? Which part of the transform does it affect?

The process where the LUT is made implies doing color transformations in a space that is not linear leading to innacurate color output. My ACESFilm LUT doesn’t do any color transformations only the tonemapper part, whereas the ACESFilm for ACEScg LUTs do the conversions in linear space so they are the recommended method (until OCIO that is).

For the same reasons explained above, I don’t recommend tweaking your exposure to compensate for dark textures from the “Utility - sRGB - Texture” IDT, lookdev normally is done in rather flat lighting. As always I recommend to use gamma (for the time being). Long story short; exposure brightens bright values, gamma brightens dark values.

20

We are starting a Virtual Production project that includes building lighting tools and standardized assets (buildings, environments, props) in UE that would be easily re-usable in other VP projects. We see ACES as a critical element for this workflow to tie the color spaces together. As we try to use off the shelf assets we are finding more and more that tricks are frequently used to get a UE asset to work in one specific project . It often requires modifications to blueprints and textures to make them portable into new VP projects using different lighting and combining them with other assets.
ACES AND VP Questions:

  1. Is anyone working on a standard for ACES compatible UE assets for use in Virtual Production?
  2. ACES and VP with UE would require a defined workflow as well? I think the asset standard would need to be tied to a recommended workflow. Is this a project being considered?
  3. I assume an ODT would be needed for the LED Wall and an IDT for the camera - would these need to be developed specifically for this VP process or will a standard camera IDT and display ODT work?
  4. For VP lens calibration is also a critical component - dont know if that should be part of the workflow.
  5. How do you see the virtual screen color using an ODT, the camera with an IDT interacting with the live actors and props on stage?
    Exciting times to be in our industry. Look forward to hearing any comments/suggestions.
1 Like