Oiiotool for calculating the proposed error metric

Discussions - ACESNext VWG – Common LUT Format (CLF)
1

In the CLF VWG we have discussed the need for an easy-to-use tool that CLF implementors could use to measure the error on output from their software or hardware, relative to some reference image that has undergone processing of the same CLF file(s). The proposed error metric is:

abs(actual - aim) / max(abs(aim), 0.01)

where ‘actual’ is the processed output image from the CLF implementor’s software or hardware, and ‘aim’ is the image as output from a reference or example CLF implementation (e.g. OpenColorIO tools).

I think we can use oiiotool (from OpenImageIO) to calculate the results of the error metric. TBD are thresholds for max error and mean error, but believe that oiiotool will be able to handle those bits as well.

Here’s the Linux oiiotool command and args to generate an error metric image:

$ oiiotool <aim-image> <actual-image> --absdiff <aim-image> --abs --maxc 0.01 --div -o out.exr

Please give it a try if you can, and let me know what you think!

2 Likes
2

Thanks for proposing oiiotool Chris, I think that will work well. It seems to do everything other than make avocado toast (and maybe even that, once I find the right command!).

I’ve done some more work on how to avoid the NaN and Inf values from messing up the comparison and came up with some command options to do that. Turns out oiiotool has a “box” command that will set a given region of pixels to zero.

So here is the proposed command to compare two images and print out the result.

oiiotool <aim-image> --dup <actual-image> --absdiff --swap --abs --maxc 0.1 --div --box:color=0,0,0:fill=1 1008,771,1023,798 --box:color=0,0,0:fill=1 0,1023,1,1023 --fixnan error --rangecheck 0,0,0 .002,.002,.002 -o /tmp/tmp.exr

This will print something like this:

       0  < 0,0,0
       0  > .002,.002,.002
 1048576  within range

If the result says there are 0 pixels greater than 0.002, the test passed. If there are any NaNs or Infs other than the expected ones, the command will print an error message rather than those results, so that is also a failure obviously.

Based on a fair amount of testing, I’ve also decided we should increase the normalization limit from 0.01 to 0.1. I’ll go into more detail about that as well as explain the oiiotool details at our Tuesday meeting.

Doug

1 Like