Performance of Vulkan backend looks amazing #11918
foldl
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have just updated chatllm.cpp to use
ggml
from the last commit (0f2bbe6). Performance of Vulkan backend looks amazing: it is much faster than the CUDA backend (in this test to be precious).Command line options:
-m qwen2.5-1.5b.bin -ngl all -t 0 -p "write a quick sort function in python" --max_length 200
Hardware: 2080TI with 22GB.
Beta Was this translation helpful? Give feedback.
All reactions