Biased samping
This release adds the ability to modify logits before the sampling phase for each expression. You can now enforce that a field is only numbers, or that it's far more likely to be 'nun-chucks'.
The naive way of implementing this didn't add much accuracy to my desired inference outcomes - it turns out that we can't just encode our desired strings and modify the logits of those tokens, we have to encode them as they would appear in the existing (and upcoming) completion. I figured there were two main uses for modifying logits - one is to modify the odds of specific sequences, the other is to target specific character classes (eg. numbers). These require slightly different handling in terms of figuring out the desired tokens -- I'm not sure I got it right in this first pass, especially with character classes, but I'll keep improving it as I bump into bugs.
There's also an auto-generated docs site now: https://gsuuon.github.io/ad-llama
Hope it makes the API easier to understand!