Can't change n_ctx?

[mlsterpr0]

On android, n_ctx and n_predict seem to always be at 512, according to app settings console, although I'm setting it higher. Is it a bug or just my budget phone? Gemma 2 2b works great, just stops generating after context length reaching 512 (i guess?). My phone has 6gb ram.

[danemadsen]

yeah sounds like a bug, ill look into it

[mlsterpr0]

From memory its meant to be model specific. So if the model doesn't support a longer context length it drops back to 512

The model is newest gemma 2 2b. Must be 8192, not sure. Same gguf file works in windows but not on android. I mean n_ctx. Actually, i noticed this problem some months ago, doesn't matter which model.

[danemadsen]

weird. Where did you download the apk from? GPlay, FDroid or here?

[mlsterpr0]

weird. Where did you download the apk from? GPlay, FDroid or here?

Doesn't matter. Both here and gplay. Who knows, maybe it's my "slightly old" budget phone poco x3 nfc on Android 12 :)

[danemadsen]

okthx for the info ill look into it for you when i get time

[mlsterpr0]

okthx for the info ill look into it for you when i get time

update: Actually, i just noticed, even windows version doesn't want n_ctx to be anything than 512. I was able to change it to 4096 minutes ago(according to console log), but now i can't :) No idea how it worked one time, but not anymore. Will try to find out how i did it:) Maybe the same strategy will work on Android.

[jensen1207]

On android, n_ctx and n_predict seem to always be at 512, according to app settings console, although I'm setting it higher. Is it a bug or just my budget phone? Gemma 2 2b works great, just stops generating after context length reaching 512 (i guess?). My phone has 6gb ram.

Hi mlsterpr0
How about your gemma 2 2b response speed? Mine is very slow.

[mlsterpr0]

Don't want to advertise here... But have you tried "layla lite" app? You can load your local gguf model like gemma too.

[danemadsen]

I'd prefer if people didn't advertise paid / proprietary apps but if you're looking for a different mobile front-end there's chatter ui

https://github.com/Vali-98/ChatterUI

[jensen1207]

I'm very confused that you recommend other product in your project page

[danemadsen]

I'm very confused that you recommend other product in your project page

What do you mean? The only thing I link to on the main page are open source projects maid depends on or projects maid has benifited greatly from.