Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xinference 加载的rerank模型 添加进来后无法调用 #794

Open
sinopec opened this issue Feb 26, 2025 · 1 comment
Open

xinference 加载的rerank模型 添加进来后无法调用 #794

sinopec opened this issue Feb 26, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@sinopec
Copy link

sinopec commented Feb 26, 2025

例行检查

  • [x ] 我已确认目前没有类似 issue
  • [ x] 我已确认我已升级到最新版本
  • [ x] 我已完整查看过项目 README,尤其是常见问题部分
  • [x ] 我理解并愿意跟进此 issue,协助测试和提供反馈
  • [x ] 我理解并认可上述内容,并理解项目维护者精力有限,不遵循规则的 issue 可能会被无视或直接关闭

问题描述
通过xinference 加载的rerank模型bge-reranker-v2-m3 ,使用curl脚本测试正常。

但是通过渠道添加进new-api后,使用同样的脚本,调用返回
{"error":{"message":"not implemented (request id: 202502261550548185154107K8AGBRS)","type":"new_api_error","param":"","code":"convert_request_failed"}}

查看后台log,记录如下:
new-api | [SYS] 2025/02/26 - 16:32:33 | model ratio not found: bge-reranker-v2-m3
new-api | [INFO] 2025/02/26 - 16:32:33 | 20250226163233233467847qyMQRCIm | user 1 with unlimited token has enough quota $199.710980, trusted and no need to pre-consume
new-api | [GIN] 2025/02/26 - 16:32:33 | 20250226163233233467847qyMQRCIm | 500 | 765.567µs | 10.30.0.123 | POST /v1/rerank
new-api | [ERR] 2025/02/26 - 16:32:33 | 20250226163233233467847qyMQRCIm | relay error (channel #2, status code: 500): not implemented (request id: 20250226163233233467847qyMQRCIm)

复现步骤
xinference 启动 rerank模型,
将xinference渠道添加new-api
使用下面脚本测试rereank功能
curl -X 'POST'
'http:/10.255.114.31:3000/v1/rerank'
-H 'accept: application/json'
-H 'Content-Type: application/json'
-H 'Authorization: Bearer sk-aaaa'
-d '{
"model": "bge-reranker-v2-m3",
"query": "A man is eating pasta.",
"documents": [
"A man is eating food.",
"A man is eating a piece of bread.",
"The girl is carrying a baby.",
"A man is riding a horse.",
"A woman is playing violin."
]
}'

预期结果

如同直接访问xinfeence端点一样正常返回结果
{
"id": "9c815c12-f45b-11ef-a5a9-bff6a7a8f883",
"results": [
{
"index": 0,
"relevance_score": 0.999574601650238,
"document": null
},
{
"index": 1,
"relevance_score": 0.07814765721559525,
"document": null
},
{
"index": 3,
"relevance_score": 1.7700713215162978e-05,
"document": null
},
{
"index": 2,
"relevance_score": 1.63753629749408e-05,
"document": null
},
{
"index": 4,
"relevance_score": 1.6318974303430878e-05,
"document": null
}
],
"meta": {
"api_version": null,
"billed_units": null,
"tokens": null,
"warnings": null
}
}

相关截图

渠道设置如图:
Image

@sinopec sinopec added the bug Something isn't working label Feb 26, 2025
@Calcium-Ion
Copy link
Owner

你用openai渠道类型接入当然不能用了,况且我们也没有兼容xinference的rerank

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants