Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

增加deepseek-r1-distill-qwen量化版本的支持 #2824

Open
kingdomad opened this issue Feb 8, 2025 · 1 comment
Open

增加deepseek-r1-distill-qwen量化版本的支持 #2824

kingdomad opened this issue Feb 8, 2025 · 1 comment
Labels
Milestone

Comments

@kingdomad
Copy link

Feature request / 功能建议

增加下列模型的内置支持
tclf90/deepseek-r1-distill-qwen-32b-gptq-int8
tclf90/deepseek-r1-distill-qwen-14b-gptq-int4

或增加一个功能,可以配置内置模型的model_specs,这样就可以自己添加新的量化版本了。

Motivation / 动机

增加某个内置模型的量化版本太麻烦了

Your contribution / 您的贡献

@XprobeBot XprobeBot added this to the v1.x milestone Feb 8, 2025
@qinxuye
Copy link
Contributor

qinxuye commented Feb 8, 2025

{
"model_format": "gptq",
"model_size_in_billions": 7,
"quantizations": [
"Int4"
],
"model_id": "tclf90/deepseek-r1-distill-qwen-7b-gptq-int4",
"model_hub": "modelscope"
},

可以在这个文件添加,欢迎提交 PR。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants