Rawtoaces - Calling All “developer-types”

@cinelogdcp, yes it appears that data must be 5nm. I think all internal calculations in rawtoaces are done at 5nm increments. The error message is not as descriptive as it could be, and better documentation would help make it clearer the current constraints around imported data.

I have opened an issue on Github on your behalf related to this https://github.com/ampas/rawtoaces/issues/100

Hi Fancois,

We will consider the two features in the next version. As you may have known, ACES file should not be compressed according to the specification. In the meanwhile, we can provide you a “ctlrender” command for you to produce compressed OpenEXR files in “ACEScg”.

You can use Photoshop to achieve it too. Please take a look at this page suggested by @kirkthibault

Please let us know if you have questions!

Thanks,

Miaoqi

Hi Miaoqi, thanks for the suggestion. This is an option, however there is an advantage to doing the exposure merging before debayering, because the noise can be significantly reduced. This is the advantage of the HDRMerge software, where the exposure merging happens before debayering. See the Zero Noise Virtual Raw article, which is a technique HDRMerge employs.

I understand if this is difficult to implement though, I don’t have complete knowledge of the complexities involved in supporting 32bpc float dng files.

Thanks for considering it!

Thanks Scott. I did work that out previously but came to the conclusion that, for accuracy reasons, extrapolating incomplete QE data sets may not be such a good idea.

Fortunately published QE data for newer, non DSLR sensors typically exceed the 380nm to 780nm bandwidth required by RawToAces so it’s just a matter of plotting waveform diagrams to JSON. I charted a few of the sensors used in DJI drone cameras and Blackmagic cameras but it’s tedious work.

Hi all,

As of 03/23/2018, the latest update of “Ceres-solver” expects the option of “max_lbfs_rank” to be set explicitly and correctly.

I have set its default value to be “50”. It may slow down the regression process a little, but it works now.

You can re-build “rawtoaces” from:

This fix, along with a few others, will be merged into AMPAS master branch soon.

Thanks and have a great weekend!

Miaoqi

1 Like

Hi there,
thanks for building great tool!

I’ve been testing the software, and I’ve noticed two thing.

1: -W option has no effect because no_auto_bright option is always 1 in the source code.

2: XYZ to ACES matrix is little different from specification.

In the official specification, CIE XYZ to ACES transform is defined as

1.04981101 75 0.00000000 00 - 0.0000974845
-0.49590302 31 1.37331304 58 0.09824003 61
0.00000000 00 0.00000000 00 0.99125201

In lib/define.h,
it is defined as

static const double XYZ_acesrgb_3[3][3] = {
{ 1.0634731317028, 0.00639793641966071, -0.0157891874506841 },
{ -0.492082784686793, 1.36823709310019, 0.0913444629573544 },
{ -0.0028137154424595, 0.00463991165243123, 0.91649468506889 }
};

and it is being applied to the result of raw image converted to XYZ D60 colorspace when we are converting non-DNG raw file,

From the name, I am assuming that XYZ_acesrgb is a matrix that converts image from XYZ to ACES colorspace, but is there some transform applied to the conversion matrix that I am not aware of?

Thanks!

Hello,
I started investigating this because I noticed developed ACES images seems to be all shifted toward blue,
and here’s some update on what I found.

It seems like we are using a matrix is a XYZ to ACES transform with Bradford Matrix from D65 to D60 applied.

So there are two chromatic adaptation applied when rendering Non-DNG files.
(CAT02 matrix from D50 to D60, and Bradford Matrix from D65 to D60),
and it doesn’t really make sense to have two chromatic adaptation to D60 applied twice.

Since the code is using libraw to convert to XYZ colorspace,
it seems that converted XYZ is in D65, and not D50,
so the first CAT02 adaptation from D50 to D60 is not needed in this case.

When I applied inverse of CAT02 to the developed ACES images,
the result I got was closer to what I expected.
(Since matrix multiplication is not commutative,
I know it’s not a correct result, but the result seems more closer)

Can anyone from ACES team take a look into it?

@kisakata Thanks so much for digging into this! Can you add issues to github so we can keep them on the list of things to fix?

Thanks so much

@Alexander_Forsythe Thanks for the quick reply.

I’ve created a pull request that reflects the change that I made while testing.
(https://github.com/ampas/rawtoaces/pull/108)
Hope my comments are enough to figure out the issue is.

Would be nice if someone from ACES take a look at it,
because I’m not really certain as to what is going on inside the code.

Thanks!

Hi

If my camera (Fuji X-T20) doesnt have any spectral sensitivity info, is there then any point in using rawtoaces over libraw?

Libraw (0.19.0) is able to pick up the white balance multipliers from the raw files, but rawtoaces doesnt?

thanks

@laserpanda yes! Rawtoaces defaults to building an IDT based on spec sens if it’s available but if not it will used metadata from the file or libraw to build an IDT. Your mileage may vary but it’s worth a try.

I’ve created a pull request that reflects the change that I made while testing.
(https://github.com/ampas/rawtoaces/pull/108 )
Hope my comments are enough to figure out the issue is.

Would be nice if someone from ACES take a look at it,
because I’m not really certain as to what is going on inside the code.

Following up on this, I notice the blue tint issue with DNG conversions is still open on Github. I can verify I’ve had what I think is the same issue.

Attached are five raw conversions for a CR2 shot with a 5D Mk II. The last shows the blue cast for DNG conversions.

  1. Rawtoaces mat-method 0 conversion of CR2 using the 5D MkII spectral sensitivities included with the rawtoaces installation. I consider this to be the baseline as I imagine it’s the most accurate.

  2. Rawtoaces mat-method 1 conversion of CR2 using metadata. Still quite similar to the spectral sensitivities conversion but a bit cooler.

  3. Canon DPP CR2 to jpeg conversion, then imported to Nuke ACES working space using utility sRGB. Then some slight luminance-only gamma and exposure adjustment to better match example 1.

  4. DCRaw output as ACES gamut linear tiff, flags are dcraw -4 -o 6 -H 3 -w -T
    Exposure lifted by +4 stops in Nuke to match the exposure of the rawtoaces spectral sensitivities conversion. I tried both CR2 and CR2 converted to DNG with Adobe’s DNG Converter as inputs for DCRaw. I got the same visual result for both CR2 and DNG, hence only attaching one image. Overall it’s similar, and warm.

  5. Rawtoaces mat-methods 1 and 2 for the same DNG I used in example 4 (converted from CR2).

I get the same result for both methods. You can see the image is much more blue which backs up the issue first found by kisakata. I’m showing one example here, but I’ve had consistently the same issue with DNGs with other devices.

I’ve visited the pull request page and the last status update on July 17th was “We’ve confirmed that the XYZ_acesrgb matrix is in fact using a D65 to D60 Bradford matrix. Your code changes are being verified by the admin, to be updated to the source master.”

It would be great to see this fix folded into the version I would install via homebrew. Thanks so much.

(Edited to fix wrong image uploaded and correcting a few things)

1 Like

For all windows users out there, You can install this software using WSL (Windows Subsystem for Linux) inside Windows 10! Just follow the instructions on the github for Ubunut and you’re fine.

I have also created two batch files to help you install this on your windows machine and later convert raw files by draging them into the other batch file.
You can download the batch files here: https://github.com/ampas/rawtoaces/issues/116

  1. Install the ubuntu distro from the windows store
  2. Launch it onces to setup username and password
  3. Run InstallRawToAces.bat and let it run
  4. Drag your raw files to the RawToAces.bat to convert them to ACES exr files.
2 Likes

Thanks Jacob for your first post and this contribution!

Steve T

Hello,

I’ve want to use rawtoaces for out textures workflow, but can’t figure out where to get camera spectral data for cameras that are not included in default build? I need Canon EOS 5D mk III + IV, and NIKON D850, but have no idea where to get them.

I was thinking about just reusing data from previous models (eg 5D mk II and D700), but I have no way of telling if spectral data are same or not. Any ideas what to do, what would be best approach?

1 Like

Welcome Jan and thanks for your first post. Quite sure someone will answer you as soon as they get settled in from the holiday!
Steve T
ACESCentral Admin

Help me somebody for install rawtoaces please! I’m just an animator :frowning:
I stopped on this part:
brew install ceres-solver --HEAD
Terminal say to me:
**Applying 941ea13475913ef8322584f7401633de9967ccc8.patch**
patching file cmake/FindTBB.cmake
Hunk #1 FAILED at 429.
1 out of 1 hunk FAILED -- saving rejects to file cmake/FindTBB.cmake.rej
READ THIS: https://docs.brew.sh/Troubleshooting

If I try to install:
brew install rawtoaces
I get this:
Error: rawtoaces has been disabled because it does not build!

Please help me!

1 Like

Hi Illya,
Thanks for your first post and welcome! If you don’t get any response in a bit, I’ll try to find you some help.
Best,

ST

Hi @eoldrk,

If you have Docker, I would recommend using that Dockerfile: PR: Add "Dockerfile". by KelSolaar · Pull Request #118 · ampas/rawtoaces · GitHub

Keeps everything in the Container and it is clean. I built it again a week ago and it works.

Cheers,

Thomas

2 Likes

@eoldrk @stobenkin were you able to find a solution? I’ve had the same issue and have been unable to build for months, more specifically since updating ceres-solver.

That said @Thomas_Mansencal with docker it works great thanks so much!

1 Like