Got it.
It’s a complex subject. I guess the CIE-Yxy space wasn’t really designed with these extreme values in mind either. (do they still count as visible colors if they blind you while looking at them and you can’t distinguish them from another light source?)
In the meantime I grabbed the matrix from this post: Colour artefacts or breakup using ACES which is helping for these highlights, but is not really different from my original solution of keeping it linear AWG.
Since the color shift is too pronounced on some of the clothes in my plate I’m restoring these via a difference key so that only these crazy highlights get affected. Not ideal but it gives me the most visually pleasing results.