Hi and Welcome @mheberlein!
Without looking to your code (it is not linked) or testing, the issue you are having is (almost certainly) caused by the different whitepoints, when you use the sRGB to XYZ matrix, the adopted white point is D65 while for the XYZ to ACES2065-1, it is that of ACES, i.e. ~= D60.
The simplest solution is to generate the RGB to RGB matrix that you need, easiest way is using our App here: https://www.colour-science.org/apps/, pick CAT02 as the CAT transform and you should be good to go.
Cheers,
Thomas