mirror of
https://github.com/labring/FastGPT.git
synced 2025-07-28 09:03:53 +00:00
4.7.1-alpha2 (#1153)
Co-authored-by: UUUUnotfound <31206589+UUUUnotfound@users.noreply.github.com> Co-authored-by: Hexiao Zhang <731931282qq@gmail.com> Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>
This commit is contained in:
Archer
121
docSite/content/docs/development/custom-models/bge-rerank.md
Normal file
121
docSite/content/docs/development/custom-models/bge-rerank.md
Normal file
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: '接入 bge-rerank 重排模型'
|
||||
description: '接入 bge-rerank 重排模型'
|
||||
icon: 'sort'
|
||||
draft: false
|
||||
toc: true
|
||||
weight: 910
|
||||
---
|
||||
|
||||
## 不同模型推荐配置
|
||||
|
||||
推荐配置如下：
|
||||
|
||||
{{< table "table-hover table-striped-columns" >}}
|
||||
| 模型名 | 内存 | 显存 | 硬盘空间 | 启动命令 |
|
||||
|------|---------|---------|----------|--------------------------|
|
||||
| bge-rerank-base | >=4GB | >=4GB | >=8GB | python app.py |
|
||||
| bge-rerank-large | >=8GB | >=8GB | >=8GB | python app.py |
|
||||
| bge-rerank-v2-m3 | >=8GB | >=8GB | >=8GB | python app.py |
|
||||
{{< /table >}}
|
||||
|
||||
## 源码部署
|
||||
|
||||
### 1. 安装环境
|
||||
|
||||
- Python 3.9, 3.10
|
||||
- CUDA 11.7
|
||||
- 科学上网环境
|
||||
|
||||
### 2. 下载代码
|
||||
|
||||
3 个模型代码分别为：
|
||||
|
||||
1. [https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base)
|
||||
2. [https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-large](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-large)
|
||||
3. [https://github.com/labring/FastGPT/tree/main/python/reranker/bge-rerank-v2-m3](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-rerank-v2-m3)
|
||||
|
||||
### 3. 安装依赖
|
||||
|
||||
```sh
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 4. 下载模型
|
||||
|
||||
3个模型的 huggingface 仓库地址如下：
|
||||
|
||||
1. [https://huggingface.co/BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base)
|
||||
2. [https://huggingface.co/BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large)
|
||||
3. [https://huggingface.co/BAAI/bge-rerank-v2-m3](https://huggingface.co/BAAI/bge-rerank-v2-m3)
|
||||
|
||||
在对应代码目录下 clone 模型。目录结构：
|
||||
|
||||
```
|
||||
bge-reranker-base/
|
||||
app.py
|
||||
Dockerfile
|
||||
requirements.txt
|
||||
```
|
||||
|
||||
### 5. 运行代码
|
||||
|
||||
```bash
|
||||
python app.py
|
||||
```
|
||||
|
||||
启动成功后应该会显示如下地址：
|
||||
|
||||
|
||||
|
||||
> 这里的 `http://0.0.0.0:6006` 就是连接地址。
|
||||
|
||||
## docker 部署
|
||||
|
||||
**镜像名分别为:**
|
||||
|
||||
1. registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-base:v0.1 (4 GB+)
|
||||
2. registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-large:v0.1 (5 GB+)
|
||||
3. registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-v2-m3:v0.1 (5 GB+)
|
||||
|
||||
**端口**
|
||||
|
||||
6006
|
||||
|
||||
**环境变量**
|
||||
|
||||
```
|
||||
ACCESS_TOKEN=访问安全凭证，请求时，Authorization: Bearer ${ACCESS_TOKEN}
|
||||
```
|
||||
|
||||
**运行命令示例**
|
||||
|
||||
```sh
|
||||
# auth token 为mytoken
|
||||
docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken --gpus all registry.cn-hangzhou.aliyuncs.com/fastgpt/bge-rerank-base:v0.1
|
||||
```
|
||||
|
||||
**docker-compose.yml示例**
|
||||
```
|
||||
version: "3"
|
||||
services:
|
||||
reranker:
|
||||
image: registry.cn-hangzhou.aliyuncs.com/fastgpt/rerank:v0.2
|
||||
container_name: reranker
|
||||
# GPU运行环境，如果宿主机未安装，将deploy配置隐藏即可
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: all
|
||||
capabilities: [gpu]
|
||||
ports:
|
||||
- 6006:6006
|
||||
environment:
|
||||
- ACCESS_TOKEN=mytoken
|
||||
|
||||
```
|
||||
## 接入 FastGPT
|
||||
|
||||
参考 [ReRank模型接入](/docs/development/configuration/#rerank-接入)，host 变量为部署的域名。
|
||||
@@ -1,90 +0,0 @@
|
||||
---
|
||||
title: '接入 ReRank 重排模型'
|
||||
description: '接入 ReRank 重排模型'
|
||||
icon: 'sort'
|
||||
draft: false
|
||||
toc: true
|
||||
weight: 910
|
||||
---
|
||||
|
||||
## 推荐配置
|
||||
|
||||
推荐配置如下：
|
||||
|
||||
{{< table "table-hover table-striped-columns" >}}
|
||||
| 类型 | 内存 | 显存 | 硬盘空间 | 启动命令 |
|
||||
|------|---------|---------|----------|--------------------------|
|
||||
| base | >=4GB | >=3GB | >=8GB | python app.py |
|
||||
{{< /table >}}
|
||||
|
||||
## 部署
|
||||
|
||||
### 环境要求
|
||||
|
||||
- Python 3.10.11
|
||||
- CUDA 11.7
|
||||
- 科学上网环境
|
||||
|
||||
### 源码部署
|
||||
|
||||
1. 根据上面的环境配置配置好环境，具体教程自行 GPT；
|
||||
2. 下载 [python 文件](https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base)
|
||||
3. 在命令行输入命令 `pip install -r requirements.txt`；
|
||||
4. 按照[https://huggingface.co/BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base)下载模型仓库到app.py同级目录
|
||||
5. 添加环境变量 `export ACCESS_TOKEN=XXXXXX` 配置 token，这里的 token 只是加一层验证，防止接口被人盗用，默认值为 `ACCESS_TOKEN` ；
|
||||
6. 执行命令 `python app.py`。
|
||||
|
||||
然后等待模型下载，直到模型加载完毕为止。如果出现报错先问 GPT。
|
||||
|
||||
启动成功后应该会显示如下地址：
|
||||
|
||||
|
||||
|
||||
> 这里的 `http://0.0.0.0:6006` 就是连接地址。
|
||||
|
||||
### docker 部署
|
||||
|
||||
+ 镜像名: `registry.cn-hangzhou.aliyuncs.com/fastgpt/rerank:v0.2`
|
||||
+ 端口号: 6006
|
||||
+ 大小：约8GB
|
||||
|
||||
**设置安全凭证（即oneapi中的渠道密钥）**
|
||||
```
|
||||
ACCESS_TOKEN=mytoken
|
||||
```
|
||||
|
||||
**运行命令示例**
|
||||
- 无需GPU环境，使用CPU运行
|
||||
```sh
|
||||
docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken registry.cn-hangzhou.aliyuncs.com/fastgpt/rerank:v0.2
|
||||
```
|
||||
|
||||
- 需要CUDA 11.7环境
|
||||
```sh
|
||||
docker run -d --gpus all --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken registry.cn-hangzhou.aliyuncs.com/fastgpt/rerank:v0.2
|
||||
```
|
||||
|
||||
**docker-compose.yml示例**
|
||||
```
|
||||
version: "3"
|
||||
services:
|
||||
reranker:
|
||||
image: registry.cn-hangzhou.aliyuncs.com/fastgpt/rerank:v0.2
|
||||
container_name: reranker
|
||||
# GPU运行环境，如果宿主机未安装，将deploy配置隐藏即可
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: all
|
||||
capabilities: [gpu]
|
||||
ports:
|
||||
- 6006:6006
|
||||
environment:
|
||||
- ACCESS_TOKEN=mytoken
|
||||
|
||||
```
|
||||
## 接入 FastGPT
|
||||
|
||||
参考 [ReRank模型接入](/docs/development/configuration/#rerank-接入)，host 变量为部署的域名。
|
||||
Reference in New Issue
Block a user