[sidecar] Predicted Output integration with OpenAI

[theskcd]

Idea

We have been using Sonnet3.5 for search and replace style editing, it does make sense to try and push Predicted Output (which is another way of saying speculative editing) and see how that performs.

Have a strong gut feeling that the latency wins aside, it might be a good unlock

[redjonzaci]

how can i help with this?

[theskcd]

Hey! Thanks for your interest in implementing this.. I think to start with we can start with proof that it can work.

If you could quickly put together a binary which showcases how predicted output works in rust that would be a great start.

We can then figure out the plumbing efforts for this