using Ollama as a backend, chat works but when told to turn on/off devices it claims it works but does nothing.

[paulash]

title says it all, i'm using a local ollama api to run my LLM which works fine if i'm just asking a question but if I ask it to turn on a light for example, it 'says' it turned on the light but nothing actually happens? I'm using just the phi:latest model for now just to get things working. Any help would be appreciated!

[paulash]

I also tried the home-3b model as well now, same thing pretty much.

[colaborat0r]

Yes you get that with other models, or a modified prompt. What model did you try before?

…

On Mon, Feb 12, 2024 at 1:24 AM paulash ***@***.***> wrote: I also tried the home-3b model as well now, same thing pretty much. — Reply to this email directly, view it on GitHub <#62 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A64GEKHGPE3ZSKQBISETQRLYTGRQPAVCNFSM6AAAAABDEEMMECVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DIMZXGQZDM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[acon96]

In short: no

If you want to mess around with larger context sizes, I recommend using text-generation-webui to load your desired model and you can mess with either the RoPE settings or use a self-extend method to increase the context size of the model beyond 2048 tokens.

I did implement the check for when your model is not exceeding the context length and will let you know why the model is outputting gibberish. If you are using Ollama, the backend doesn't properly report how many tokens were in the system prompt so this feature doesn't work properly.

[ther3zz]

Gotcha... Out of curiosity, are you at least filtering out unavailable devices? I'm sure that will help shorten the context

Btw I'm using the dolphin-mixtral model and for me sometimes it works but other times it cant "find" the device... I'm going to enable logging to see what is being sent because that seems like odd behavior. I couldn't get it to work at all with the mistral model.

[acon96]

Yes, the model only uses entities that are "exposed" to the voice assistant. There is a UI for doing that on the Voice Assistant settings page.

[ther3zz]

Sorry, more specifically, does it filter out "offline" devices?
I think if an entity is exposed but it is offline, it probably still passes through. At least this way wouldn't require manual intervention to decrease the context size

[ther3zz]

I just checked debug logs real quick and it looks like "unavailable" (offline) devices/entities are being passed