Is there a simple way to add a prefix to "assistant" response without giving up LlamaChatSession?

[earslap]

First of all, thank you for the fantastic project.

I'm trying to figure out if I can add a prefix to assistant response to guide the generation better.

Imagine I'm trying to use the model for summarization. My system prompt reflects that, but some models still get lost if the provided content in the user message is too long. So to guide the generation better, I'd like the assistant generation to start with a given prefix for the next message such as "Here is the summary of the provided text:"

Right now, I can call session.prompt but seems like chat template takes over and I have no way of filling in the response of the assistant manually. Or do I?

I could just construct the messages manually using a given model's format but I was wondering if there is a way to do this without giving up the ergonomics of LlamaChatSession.

[giladgd]

You can use LlamaChat to generate a completion for a given chat history that already includes a partial model response.
I've added the ability to provide a responsePrefix option to LlamaChatSession's .prompt() function in #375, which I'll release in a few days.

Thanks for the idea :)

[earslap]

Well that was quick, thank you!

[giladgd]

I've released a new version with this feature.
You can see an example usage of it here.