speculative decoding

Signed-off-by: Prabhu Subramanian <[email protected]>
CycloneDX · Feb 8, 2025 · 266bdf4 · 266bdf4
1 parent 9615680
commit 266bdf4
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/contrib/xBOMEval/cdx1-NOTES.md b/contrib/xBOMEval/cdx1-NOTES.md
@@ -278,3 +278,5 @@ Prompt: 27 tokens, 57.950 tokens-per-sec
 Generation: 40 tokens, 19.741 tokens-per-sec
 Peak memory: 23.951 GB
 ```
+
+For the `--num-draft-tokens` argument, use a value between 2 - 4.