Customizing vectors on the fly with a task-specific matrix

[ibestvina]

What would be the best approach (if it is possible) to provide the system with a matrix M alongside a query vector q, such that the similarity by which search is compared is calculated on (qM, xM) pairs (for the x vectors in the index). This approach is suggested, and seems to work well, in the OpenAI notebook here.

I am intentionally keeping this very vague, as I know standard indexes would not support this, but maybe there is some place where this concept could neatly fit in.