-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Co-authored-by: UUUUnotfound <[email protected]> Co-authored-by: Hexiao Zhang <[email protected]> Co-authored-by: heheer <[email protected]>
- Loading branch information
1 parent
3b0b2d6
commit 1fbc407
Showing
84 changed files
with
1,772 additions
and
714 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
name: Release | ||
name: Release helm chart | ||
|
||
on: | ||
push: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
121 changes: 121 additions & 0 deletions
121
docSite/content/docs/development/custom-models/bge-rerank.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,121 @@ | ||
--- | ||
title: '接入 bge-rerank 重排模型' | ||
description: '接入 bge-rerank 重排模型' | ||
icon: 'sort' | ||
draft: false | ||
toc: true | ||
weight: 910 | ||
--- | ||
|
||
## 不同模型推荐配置 | ||
|
||
推荐配置如下: | ||
|
||
{{< table "table-hover table-striped-columns" >}} | ||
| 模型名 | 内存 | 显存 | 硬盘空间 | 启动命令 | | ||
|------|---------|---------|----------|--------------------------| | ||
| bge-rerank-base | >=4GB | >=4GB | >=8GB | python app.py | | ||
| bge-rerank-large | >=8GB | >=8GB | >=8GB | python app.py | | ||
| bge-rerank-v2-m3 | >=8GB | >=8GB | >=8GB | python app.py | | ||
{{< /table >}} | ||
|
||
## 源码部署 | ||
|
||
### 1. 安装环境 | ||
|
||
- Python 3.9, 3.10 | ||
- CUDA 11.7 | ||
- 科学上网环境 | ||
|
||
### 2. 下载代码 | ||
|
||
3 个模型代码分别为: | ||
|
||
1. [https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base) | ||
2. [https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-large](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-large) | ||
3. [https://github.com/labring/FastGPT/tree/main/python/reranker/bge-rerank-v2-m3](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-rerank-v2-m3) | ||
|
||
### 3. 安装依赖 | ||
|
||
```sh | ||
pip install -r requirements.txt | ||
``` | ||
|
||
### 4. 下载模型 | ||
|
||
3个模型的 huggingface 仓库地址如下: | ||
|
||
1. [https://huggingface.co/BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) | ||
2. [https://huggingface.co/BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) | ||
3. [https://huggingface.co/BAAI/bge-rerank-v2-m3](https://huggingface.co/BAAI/bge-rerank-v2-m3) | ||
|
||
在对应代码目录下 clone 模型。目录结构: | ||
|
||
``` | ||
bge-reranker-base/ | ||
app.py | ||
Dockerfile | ||
requirements.txt | ||
``` | ||
|
||
### 5. 运行代码 | ||
|
||
```bash | ||
python app.py | ||
``` | ||
|
||
启动成功后应该会显示如下地址: | ||
|
||
![](/imgs/rerank1.png) | ||
|
||
> 这里的 `http://0.0.0.0:6006` 就是连接地址。 | ||
## docker 部署 | ||
|
||
**镜像名分别为:** | ||
|
||
1. registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-base:v0.1 (4 GB+) | ||
2. registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-large:v0.1 (5 GB+) | ||
3. registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-v2-m3:v0.1 (5 GB+) | ||
|
||
**端口** | ||
|
||
6006 | ||
|
||
**环境变量** | ||
|
||
``` | ||
ACCESS_TOKEN=访问安全凭证,请求时,Authorization: Bearer ${ACCESS_TOKEN} | ||
``` | ||
|
||
**运行命令示例** | ||
|
||
```sh | ||
# auth token 为mytoken | ||
docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken --gpus all registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-base:v0.1 | ||
``` | ||
|
||
**docker-compose.yml示例** | ||
``` | ||
version: "3" | ||
services: | ||
reranker: | ||
image: registry.cn-hangzhou.aliyuncs.com/fastgpt/rerank:v0.2 | ||
container_name: reranker | ||
# GPU运行环境,如果宿主机未安装,将deploy配置隐藏即可 | ||
deploy: | ||
resources: | ||
reservations: | ||
devices: | ||
- driver: nvidia | ||
count: all | ||
capabilities: [gpu] | ||
ports: | ||
- 6006:6006 | ||
environment: | ||
- ACCESS_TOKEN=mytoken | ||
``` | ||
## 接入 FastGPT | ||
|
||
参考 [ReRank模型接入](/docs/development/configuration/#rerank-接入),host 变量为部署的域名。 |
90 changes: 0 additions & 90 deletions
90
docSite/content/docs/development/custom-models/reranker.md
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.