Can't provide stop tokens/patterns

[jameshfisher]

This lib does a great job of downloading and running the model! But it provides a very restricted API for interacting with it.

Here's the type signature for prompt. Comparing to other LLMs, I expect some other params, e.g. stop tokens and temperature. At the very least, my use-case requires stop tokens.

[hpxiong]

any updates on this? when I created a langchain with gpt4all model, it never stops at the given keyword.

    llm = GPT4All(
        model=model, 
        max_tokens=model_config['max_tokens'], 
        n_predict=model_config['n_predict'], 
        n_threads=model_config['n_threads'], 
        temp=model_config['temp'], 
        device=model_config['device'], 
        verbose=model_config['verbose'], 
        streaming=model_config['streaming'], 
        callbacks=callbacks,
    )

chain = prompt | llm.bind(stop=["Human:"]) | parser

sample reply: