FastGPT/document/content/docs/self-host/config/model/intro.en.mdx

---
title: Model Configuration
description: FastGPT model configuration guide
---

import { Alert } from '@/components/docs/Alert';
import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';

## Introduction

FastGPT uses the `AI Proxy` service to connect to different model providers. AI Proxy also provides load balancing, model logging, and analytics dashboards to help you monitor model usage.

<Alert icon="🤖" context="success">
  Notes:

  1. Only one speech recognition model can be active at a time, so you only need to configure one.
  2. The system requires at least one language model and one embedding model to function properly.
</Alert>

### Architecture Diagram

![alt text](../../../../../public/imgs/image-95.png)

### Model Types

1. Language Models - Text-based conversations; multimodal models also support image recognition.
2. Embedding Models - Index text chunks for semantic text retrieval.
3. Rerank Models - Reorder retrieval results to optimize search ranking.
4. Text-to-Speech (TTS) - Convert text to audio.
5. Speech-to-Text (STT) - Convert audio to text.


### Key Terminology

- Model ID: The value of the `model` field in the API request body. Must be globally unique.
- Model Name: The display name of the model, which can be customized.
- Model Channel: The protocol of different model providers, such as OpenAI, Anthropic, Google, etc. Most self-hosted channels follow the OpenAI protocol. A single model can be configured across multiple channels to enable load balancing.
- Custom Request URL / Key: Allows you to bypass Model Channels and send requests directly to a custom endpoint. You need to provide the full request URL and token. Generally not needed (not recommended as it's harder to manage).

## Adding Channels and Models

You can configure models from the `Account - Model Providers` page in FastGPT.

### 1. Create a Channel

Switch to the `Model Channels` tab. Note that you can only add models that already exist in `Model Configuration`. The system only includes mainstream models by default — if you need additional models, add them in `Model Configuration` first.

![aiproxy1](../../../../../public/imgs/aiproxy-1.png)

Click "Add Channel" in the top-right corner to open the channel configuration page.

![alt text](../../../../../public/imgs/image-122.png)

Using Alibaba Bailian models as an example:

![alt text](../../../../../public/imgs/image-123.png)

1. Channel Name: A display label for the channel, used for identification only.
2. Protocol Type: The API protocol for the model. Generally, select the provider that offers the model. Most providers support the OpenAI protocol, so you can also choose OpenAI as the protocol type.
3. Models: The specific models available in this channel. The system includes popular models by default. If the model you need isn't in the dropdown, click "Add Model" to [add a custom model](/docs/self-host/config/model/intro/#add-a-custom-model).
4. Model Mapping: Maps the model name in FastGPT requests to the actual model name at the provider. For example:

```json
{
  "gpt-4o-test": "gpt-4o"
}
```

In FastGPT, the model is `gpt-4o-test`, and requests to AI Proxy also use `gpt-4o-test`. When AI Proxy forwards the request upstream, the actual `model` value becomes `gpt-4o`.


5. Proxy URL: Do not enter the full model request URL. Enter the `BaseUrl` instead, and check whether `/v1` needs to be appended.
6. API Key: The API credentials obtained from the model provider. Some providers require multiple keys — follow the on-screen prompts to enter them.

Click "Add" to save. The new channel will appear under "Model Channels".

![aiproxy4](/imgs/aiproxy-4.png)

### 2. Channel Testing

You can test the channel to verify that the configured models are working properly.

![aiproxy5](/imgs/aiproxy-5.png)

Click "Model Test" to see the list of configured models, then click "Start Test".

![aiproxy6](/imgs/aiproxy-6.png)

Once testing completes, you'll see the results and response times for each model.

![aiproxy7](/imgs/aiproxy-7.png)

### 3. Enable Models

The system includes models from major providers by default. If you're not familiar with the configuration, simply click `Enable`. The `Model ID` corresponds to the `Model` in `Model Channels`.

Click "Enable" to activate the model.

|         Enable Models                    |            Model ID Mapping                  |
| ------------------------------- | ------------------------------- |
|![alt text](../../../../../public/imgs/image-92.png) | ![alt text](../../../../../public/imgs/image-124.png) |

### 4. Test Models

FastGPT provides simple tests for each model type on the UI to verify that models are working correctly. Each test sends an actual request using a template.

![alt text](/imgs/image-105.png)

## Model Configuration

### Edit Model Configuration

Click the gear icon next to a model to open its configuration. Different model types have different configuration options.

|                                 |                                 |
| ------------------------------- | ------------------------------- |
| ![alt text](/imgs/image-93.png) | ![alt text](/imgs/image-94.png) |

### Add a Custom Model

If the built-in models don't meet your needs, you can add custom models. If the `Model ID` matches an existing built-in model ID, it will be treated as a modification rather than a new model.

1. **Add via Form**

|                                 |                                 |
| ------------------------------- | ------------------------------- |
| ![alt text](/imgs/image-96.png) | ![alt text](/imgs/image-97.png) |

2. **Add via Configuration File**

If you find it tedious to configure models through the UI, you can use a configuration file instead. This is also useful for quickly replicating the configuration from one system to another.

|                                 |                                 |
| ------------------------------- | ------------------------------- |
| ![alt text](/imgs/image-98.png) | ![alt text](/imgs/image-99.png) |

<Accordions>
<Accordion title="Language Model Fields">
```json
{
  "model": "Model ID",
  "metadata": {
    "isCustom": true, // Whether this is a custom model
    "isActive": true, // Whether the model is enabled
    "provider": "OpenAI", // Model provider, used for categorization. Built-in providers: https://github.com/labring/FastGPT/blob/main/packages/global/core/ai/provider.ts. You can submit a PR for new providers, or use "Other"
    "model": "gpt-5", // Model ID (corresponds to the model name in the channel)
    "name": "gpt-5", // Display name
    "maxContext": 125000, // Maximum context length
    "maxResponse": 16000, // Maximum response length
    "quoteMaxToken": 120000, // Maximum citation content tokens
    "maxTemperature": 1.2, // Maximum temperature
    "charsPointsPrice": 0, // Credits per 1k tokens (commercial edition)
    "censor": false, // Enable content moderation (commercial edition)
    "vision": true, // Supports image input
    "toolChoice": true, // Supports tool selection (used in classification, extraction, and tool calls)
    "functionCall": false, // Supports function calling (used in classification, extraction, and tool calls). toolChoice takes priority; if false, functionCall is used; if also false, prompt mode is used
    "customCQPrompt": "", // Custom text classification prompt (for models without tool/function call support)
    "customExtractPrompt": "", // Custom content extraction prompt
    "defaultSystemChatPrompt": "", // Default system prompt included in conversations
    "defaultConfig": {}, // Default config sent with API requests (e.g., GLM4's top_p)
    "fieldMap": {} // Field mapping (e.g., o1 models need max_tokens mapped to max_completion_tokens)
  }
}
```
</Accordion>

<Accordion title="Embedding Model Fields">

```json
{
  "model": "Model ID",
  "metadata": {
    "isCustom": true, // Whether this is a custom model
    "isActive": true, // Whether the model is enabled
    "provider": "OpenAI", // Model provider
    "model": "text-embedding-3-small", // Model ID
    "name": "text-embedding-3-small", // Display name
    "charsPointsPrice": 0, // Credits per 1k tokens
    "defaultToken": 512, // Default token count for text splitting
    "maxToken": 3000 // Maximum token count
  }
}
```

</Accordion>

<Accordion title="Rerank Model Fields">

```json
{
  "model": "Model ID",
  "metadata": {
    "isCustom": true, // Whether this is a custom model
    "isActive": true, // Whether the model is enabled
    "provider": "BAAI", // Model provider
    "model": "bge-reranker-v2-m3", // Model ID
    "name": "ReRanker-Base", // Display name
    "requestUrl": "", // Custom request URL
    "requestAuth": "", // Custom request authentication
    "type": "rerank" // Model type
  }
}
```

</Accordion>

<Accordion title="Text-to-Speech Model Fields">

```json
{
  "model": "Model ID",
  "metadata": {
    "isActive": true, // Whether the model is enabled
    "isCustom": true, // Whether this is a custom model
    "type": "tts", // Model type
    "provider": "FishAudio", // Model provider
    "model": "fishaudio/fish-speech-1.5", // Model ID
    "name": "fish-speech-1.5", // Display name
    "voices": [
      // Available voices
      {
        "label": "fish-alex", // Voice name
        "value": "fishaudio/fish-speech-1.5:alex" // Voice ID
      },
      {
        "label": "fish-anna", // Voice name
        "value": "fishaudio/fish-speech-1.5:anna" // Voice ID
      }
    ],
    "charsPointsPrice": 0 // Credits per 1k tokens
  }
}
```

</Accordion>

<Accordion title="Speech-to-Text Model Fields">

```json
{
  "model": "whisper-1",
  "metadata": {
    "isActive": true, // Whether the model is enabled
    "isCustom": true, // Whether this is a custom model
    "provider": "OpenAI", // Model provider
    "model": "whisper-1", // Model ID
    "name": "whisper-1", // Display name
    "charsPointsPrice": 0, // Credits per 1k tokens
    "type": "stt" // Model type
  }
}
```

</Accordion>

</Accordions>

## Other

### Channel Priority

Range: 1–100. Higher values are prioritized.

![aiproxy9](/imgs/aiproxy-9.png)

### Enable / Disable Channels

In the control menu on the right side of each channel, you can enable or disable it. Disabled channels will no longer serve model requests.

![aiproxy10](/imgs/aiproxy-10.png)

### Model Call Logs

Model calls made through channels are logged on the `Call Logs` page. Logs include input/output tokens, request time, latency, request URL, and more. Failed requests show detailed parameters and error messages for debugging, but logs are retained for only 1 hour by default (configurable via environment variables).

![aiproxy11](/imgs/aiproxy-11.png)


### Self-Hosted Models

[See the ReRank model deployment tutorial](/docs/self-host/custom-models/bge-rerank/)

### Custom Request URL

If you set a custom request URL, requests will bypass `Model Channels` and be sent directly to the specified endpoint. You must provide the full request URL, for example:

- LLM: [host]/v1/chat/completions
- Embedding: [host]/v1/embeddings
- STT: [host]/v1/audio/transcriptions
- TTS: [host]/v1/audio/speech
- Rerank: [host]/v1/rerank

The custom request key is included as the `Authorization: Bearer xxx` header when sending requests to the custom URL.

All endpoints follow the OpenAI model format. Refer to the [OpenAI API documentation](https://platform.openai.com/docs/api-reference/introduction) for details.

Since OpenAI does not provide a Rerank model, the Rerank endpoint follows the Cohere format. [See request examples](/docs/self-host/faq/#how-to-troubleshoot-model-issues)

### Adding Model Presets

You can find model provider configuration files in the `modules/model/provider` directory of the `FastGPT-plugin` project and add model configurations there. Make sure the `model` field is unique across all models. For field descriptions, refer to [Model Configuration Fields](/docs/self-host/config/model/intro/#add-via-configuration-file).


### Migrating from OneAPI to AI Proxy

If you were using OneAPI in an older version, you can migrate your channel configuration to AI Proxy using a script.

Send the following HTTP request from any terminal. Replace `{{host}}` with the AI Proxy address and `{{admin_key}}` with the `ADMIN_KEY` value in AI Proxy.

The `dsn` parameter in the request body is the MySQL connection string for OneAPI.

```bash
curl --location --request POST '{{host}}/api/channels/import/oneapi' \
--header 'Authorization: Bearer {{admin_key}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "dsn": "mysql://root:s5mfkwst@tcp(dbconn.sealoshzh.site:33123)/mydb"
}'
```

A successful response will return `"success": true`.

Note that the migration script performs a simple data mapping — it primarily transfers `proxy URLs`, `models`, and `API keys`. Manual verification after migration is recommended.