mirror of
https://github.com/labring/FastGPT.git
synced 2026-04-26 02:07:28 +08:00
Archer
87b0bca30c
* cloud doc * doc refactor * doc move * seo * remove doc * yml * doc * fix: tsconfig * fix: tsconfig
125 lines
5.1 KiB
Plaintext
125 lines
5.1 KiB
Plaintext
---
|
|
title: Connect Models via OneAPI
|
|
description: Connect Models via OneAPI
|
|
---
|
|
|
|
FastGPT uses a model-separated deployment approach. FastGPT only supports the OpenAI model specification (models not available in OpenAI follow a more generic specification), and uses [One API](https://github.com/songquanpeng/one-api) to unify access to different model interfaces.
|
|
|
|
[One API](https://github.com/songquanpeng/one-api) is an OpenAI API management and distribution system that provides access to all LLMs through the standard OpenAI API format, ready to use out of the box.
|
|
|
|
## Relationship Between FastGPT and One API
|
|
|
|
Think of One API as a gateway. The relationship between FastGPT and One API:
|
|
|
|
|
|
|
|
## Deployment
|
|
|
|
### Sealos Version
|
|
|
|
* Beijing region: [Deploy OneAPI](https://hzh.sealos.run/?openapp=system-template%3FtemplateName%3Done-api)
|
|
* Singapore region (GPT available): [Deploy OneAPI](https://cloud.sealos.io/?openapp=system-template%3FtemplateName%3Done-api&uid=fnWRt09fZP)
|
|
|
|
|
|
|
|
After deployment, open the OneAPI access link to proceed with the next steps.
|
|
|
|
## OneAPI Basic Tutorial
|
|
|
|
### Concepts
|
|
|
|
1. Channels:
|
|
1. In OneAPI, a channel corresponds to one `Api Key`, which can be from GPT, Microsoft, ChatGLM, ERNIE Bot, etc. A single `Api Key` can typically call multiple models from the same provider.
|
|
2. One API routes requests to channels based on the `model` specified in the request. If a model maps to multiple channels, one is selected at random.
|
|
2. Tokens: Credentials required to access One API. You only need this single credential to access all models configured in One API. In FastGPT, you only need to configure the `baseurl` and `token` from One API. Do not set any model scope restrictions on the token, as this can cause errors.
|
|
|
|
|
|
|
|
### General Workflow
|
|
|
|
1. Client sends a request to One API.
|
|
2. One API matches the request to a channel based on the `model` parameter (must match exactly with the models in the channel). If multiple channels match, one is selected at random (same priority).
|
|
3. One API sends the request to the actual endpoint.
|
|
4. One API returns the result to the client.
|
|
|
|
### 1. Log in to One API
|
|
|
|
|
|
|
|
### 2. Create a Channel
|
|
|
|
Add a channel in One API. Click [Add Base Models] directly — don't forget the embedding models.
|
|
|
|
|
|
|
|
### 3. Create a Token
|
|
|
|
| | |
|
|
| --- | --- |
|
|
|  |  |
|
|
|
|
### 4. Adjust Account Balance
|
|
|
|
One API defaults to $200 for the root user. You can edit this as needed.
|
|
|
|
|
|
|
|
### 5. Update FastGPT Environment Variables
|
|
|
|
With the One API token, FastGPT can send requests to One API by modifying the `baseurl` and `key`, and One API will forward them to the appropriate models. Update these two environment variables:
|
|
|
|
```bash
|
|
# Make sure to include v1. If on the same network, you can use the internal address.
|
|
OPENAI_BASE_URL=https://xxxx.cloud.sealos.io/v1
|
|
# The key below is the token provided by One API
|
|
CHAT_API_KEY=sk-xxxxxx
|
|
```
|
|
|
|
## Connect Other Models
|
|
|
|
**Example: Adding ERNIE Bot:**
|
|
|
|
### 1. Add a Model Channel in OneAPI
|
|
|
|
Select "Baidu Wenxin Qianfan" as the type.
|
|
|
|
|
|
|
|
### 2. Update FastGPT Model Configuration
|
|
|
|
Open FastGPT model configuration and enable the Wenxin Qianfan model. If the model is not built-in, you can add it manually through the "Add Model" option.
|
|
|
|
|
|
|
|
## Other Provider Integration References
|
|
|
|
This section covers tutorials for connecting various providers to OneAPI. After configuration, don't forget to enable the models in FastGPT's model configuration.
|
|
|
|
### Alibaba Qwen (Tongyi Qianwen)
|
|
|
|
Qwen is now compatible with the GPT format and can be connected directly using the OpenAI type. As shown below, select `OpenAI` as the type and fill in the Alibaba Cloud proxy address.
|
|
|
|
You can directly use Alibaba Cloud's language models and the `text-embedding-v3` embedding model (confirmed to be normalized and ready to use).
|
|
|
|
|
|
|
|
### SiliconCloud — Open Source Model Collection
|
|
|
|
[SiliconCloud](https://cloud.siliconflow.cn/i/TR9Ym0c4) is a platform specializing in open source model inference with its own acceleration engine. It offers a wide range of models, making it ideal for low-cost testing of open source models. Integration steps:
|
|
|
|
1. [Register a SiliconCloud account](https://cloud.siliconflow.cn/i/TR9Ym0c4)
|
|
2. Go to the console and get your API key: https://cloud.siliconflow.cn/account/ak
|
|
3. Add a new OneAPI channel, select `OpenAI` as the type, set the proxy to `https://api.siliconflow.cn`, and use the API key from step 2.
|
|
|
|
|
|
|
|
Since OneAPI doesn't include SiliconCloud model names by default, you can enter custom model names. Here's how to find the model names:
|
|
|
|
1. Open the [SiliconCloud model list](https://siliconflow.cn/zh-cn/models)
|
|
2. Click on a model to open its details.
|
|
3. Copy the model name into OneAPI.
|
|
|
|
| | | |
|
|
| --- | --- | --- |
|
|
|  | |  |
|