Model is confused about services

[tannisroot]

Hi. I'm using Ollama API backend (Ollama version is 0.1.27, running as a TrueNAS app) and running the fixt/home-3b-v2:q8_0 model. The configuration is all default, I haven't modified the prompt.
The LLM works fine for operating lights, however, whenever I ask it to operate any other type of a device, for example, a climate entity, it gets confused and starts to make up its own invalid services, basing them off the entity's name. For example:

And with media entity, it seems to produce a response like this, which also doesn't do anything:

I've tried to modify a prompt with this additional sentence "You CANNOT make up your own services. You CANNOT add parts of a device's to the name of the service. You can ONLY use services listed below." but it still goes ahead and makes up its own services.
Finally, I added an additional sentence that gives it an example of what not to do: "For example, if I tell you to set the temperature on a boiler, you may ONLY use the "climate.set_temperature" service. You cannot MODIFY it in ANY WAY." and that didn't really work either. It just outputs this, but the entity is not actually changed.

I understand you can only do so much without changing the model, but does anyone have suggestions how to combat this and make the LLM behave? Thanks.

[tannisroot]

Got the brand new https://ollama.com/fixt/home-3b-v3 model and even the q5 one more than satisfies my tasks! There are other issues with it but it works ok for now.

[acon96]

Yeah I really need to publish the Ollama model myself when I publish to HF. I'll figure that out for the v4 model (when it comes). And for anything that doesn't work well: you can add a comment about it in the "Model Capabilities" discussion thread and I'll add it to the list of things to work on for v4.