Just following on from something I said in the meeting. I don’t believe there is actually a sudden change in the shape of the top gamut hull at the yellow hue. I think that because there is a flattened section near the cusp, a large gamma value is needed to contain that flat section. As the flat section reduces in size the large gamma is still needed, but at a certain hue the flat section reduces to nothing, and suddenly a much smaller gamma is able to contain the top shape. But the actual true shape does not change dramatically at that point.
One question is whether we actually need that large gamma. Is the cusp smoothing enough to contain the flat section anyway? Our search for the fit currently doesn’t include cusp smoothing, But could it?
The first version of that did include it (v051 I think) and was then removed to have no cusp smoothing (v052 ). But that was also before the intersection was using horizontal projection (v053) in the approximation. That meant that it wasn’t particularly accurate. Now that it uses horizontal projection the intersection should be accurate (or at least much more accurate). So perhaps it should be tested now with cusp smoothing. In the original version it made the inverse unusable.
Edit: Quick test shows promising. I just added the cusp smoothing to the approximation and made no other changes. The inverse is worse, but nothing compared to what it was when the smoothing was used originally. That one exploded completely in the inverse. So I considering this promising. The yellow is the difficult one again, so slightly larger gamma needed for that to invert.
Rec.709 BT.1886 inverse with cusp smoothing in top gamma approximation:
Edit2: I think the issue with using the cusp smoothing (or at least the same cusp smoothing value as in the mapper) is that it’s now the actual boundary for the intersection. But any inaccuracy in that boundary intersection then may mean that we’d have to add more cusp smoothing to get it inverse cleanly. So I don’t think it can be the same value. I also tested smoothCusps* 0.3 and that still wasn’t good enough, so it’s getting closer to 0 again…
What if when solving for the gamma fit we didn’t use the full smoothing, but did move the cusp out by the same amount that it is moved by when applying smoothing? That might remove the necessity for the large increase in gamma to contain the flattened segment around yellow.
Now I realise it’s not quite that simple, because the smooth minimum function erodes the cusp back in again a bit. But does it erode it all the way back to its original position? And the smooth minimum is applied after the two intersections are found, so the intersection with a gamma line from the moved cusp is what is solved for in the gamut compression.
That original experiment was only in Python for the plot. I just did a quick test of adding the same cusp move to the Blink. It seems to work on initial testing. But I need to bump the cusp smoothing up to 0.35 for a good round trip (a couple of yellow values get lost if I don’t).
The effect of these changes I can only see by looking at the scopes with our most extreme synthetic images. They are not visible to my eyes in the images.