-
I am integrating KoboldCpp into my own frontend. The initial setup works well so far. Now I want to add sampler controls to my UI but being a beginner in LLMs, I have some doubts about interpreting the documentation about samplers: It says that the default (and the best) sampler order is [6,0,1,3,4,2,5]. But what are the actual samplers corresponding to those seven list indexes? In the Swagger schemas, I see that all the sampler values are optional. But what happens if I don't send a specific value at all or send null? Would it be treated as "disabled" in KoboldCpp or would it assume some other "recommended default" value? Thank you. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 2 replies
-
0=top_k, 1=top_a, 2=top_p, 3=tfs, 4=typ, 5=temp, 6=rep_pen For details on each sampler, please refer to https://github.com/LostRuins/koboldcpp/wiki#what-are-samplers-how-do-i-change-or-disable-them-what-are-the-best-samplers Generally you don't need to (and should not) change them unless you have to. If you don't send it, the optimal default settings will be used. |
Beta Was this translation helpful? Give feedback.
-
@LostRuins Thank you. To have a specific example, let's say I call /generate API and do not send min_p nor temperature in the JSON request at all. Would KoboldCpp apply its own optimal defaults to those samplers or would the samplers be essentially disabled (not affecting LLMs output at all)? |
Beta Was this translation helpful? Give feedback.
0=top_k, 1=top_a, 2=top_p, 3=tfs, 4=typ, 5=temp, 6=rep_pen
For details on each sampler, please refer to https://github.com/LostRuins/koboldcpp/wiki#what-are-samplers-how-do-i-change-or-disable-them-what-are-the-best-samplers
Generally you don't need to (and should not) change them unless you have to. If you don't send it, the optimal default settings will be used.