Files
FastGPT/docSite/content/docs/workflow/examples/dalle3.md
Archer 439c819ff1 4.8 preview (#1288)
* Revert "lafAccount add pat & re request when token invalid (#76)" (#77)

This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.

* perf: workflow ux

* system config

* Newflow (#89)

* docs: Add doc for Xinference (#1266)

Signed-off-by: Carson Yang <yangchuansheng33@gmail.com>

* Revert "lafAccount add pat & re request when token invalid (#76)" (#77)

This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.

* perf: workflow ux

* system config

* Revert "lafAccount add pat & re request when token invalid (#76)" (#77)

This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.

* Revert "lafAccount add pat & re request when token invalid (#76)" (#77)

This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.

* Revert "lafAccount add pat & re request when token invalid (#76)" (#77)

This reverts commit 83d85dfe37adcaef4833385ea52ee79fd84720be.

* rename code

* move code

* update flow

* input type selector

* perf: workflow runtime

* feat: node adapt newflow

* feat: adapt plugin

* feat: 360 connection

* check workflow

* perf: flow 性能

* change plugin input type (#81)

* change plugin input type

* plugin label mode

* perf: nodecard

* debug

* perf: debug ui

* connection ui

* change workflow ui (#82)

* feat: workflow debug

* adapt openAPI for new workflow (#83)

* adapt openAPI for new workflow

* i18n

* perf: plugin debug

* plugin input ui

* delete

* perf: global variable select

* fix rebase

* perf: workflow performance

* feat: input render type icon

* input icon

* adapt flow (#84)

* adapt newflow

* temp

* temp

* fix

* feat: app schedule trigger

* feat: app schedule trigger

* perf: schedule ui

* feat: ioslatevm run js code

* perf: workflow varialbe table ui

* feat: adapt simple mode

* feat: adapt input params

* output

* feat: adapt tamplate

* fix: ts

* add if-else module (#86)

* perf: worker

* if else node

* perf: tiktoken worker

* fix: ts

* perf: tiktoken

* fix if-else node (#87)

* fix if-else node

* type

* fix

* perf: audio render

* perf: Parallel worker

* log

* perf: if else node

* adapt plugin

* prompt

* perf: reference ui

* reference ui

* handle ux

* template ui and plugin tool

* adapt v1 workflow

* adapt v1 workflow completions

* perf: time variables

* feat: workflow keyboard shortcuts

* adapt v1 workflow

* update workflow example doc (#88)

* fix: simple mode select tool

---------

Signed-off-by: Carson Yang <yangchuansheng33@gmail.com>
Co-authored-by: Carson Yang <yangchuansheng33@gmail.com>
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>

* doc

* perf: extract node

* extra node field

* update plugin version

* doc

* variable

* change doc & fix prompt editor (#90)

* fold workflow code

* value type label

---------

Signed-off-by: Carson Yang <yangchuansheng33@gmail.com>
Co-authored-by: Carson Yang <yangchuansheng33@gmail.com>
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>
2024-04-25 17:51:20 +08:00

12 KiB
Raw Blame History

title, description, icon, draft, toc, weight
title description icon draft toc weight
Dalle3 绘图 使用 HTTP 模块绘制图片 image false true 404

OpenAI Dalle3 接口

先来看下官方接口的参数和响应值:

Body

{
    "model": "dall-e-3",
    "prompt": "A cute baby sea otter",
    "n": 1,
    "size": "1024x1024"
}

Response

{
  "created": 1589478378,
  "data": [
    {
      "url": "https://..."
    },
    {
      "url": "https://..."
    }
  ]
}

编排思路

  1. 通过 AI 来优化图片绘制的提示词(这步省略了,自己找提示词即可)
  2. 通过 【HTTP 请求】模块 调用 Dalle3 接口,获取图片的 URL。
  3. 通过 【文本加工】模块 来构建 Markdown 的图片格式。
  4. 通过 【指定回复】模块 来直接输出图片链接。

1. 构建 HTTP 模块

请求参数直接复制 Dalle3 接口的即可,并求改 prompt 为变量。需要增加一个 Headers.Authorization

Body:

{
    "model": "dall-e-3",
    "prompt": "{{prompt}}",
    "n": 1,
    "size": "1024x1024"
}

Headers:

Authorization: Bearer sk-xxx

Response:

响应值需要根据 Dalle3 接口的返回值进行获取我们只绘制了1张图片所以只需要取第一张图片的 URL 即可。给 HTTP 模块增加一个自定义输出 data[0].url

2. 文本加工 - 构建图片链接

Markdown 语法中 ![图片描述](图片链接) 表示插入图片图片链接由【HTTP 请求】模块输出。

因此可以增加一个输入来接收 【HTTP 请求】模块 的图片链接输出,并在 【文本加工】模块 - 文本 中通过变量来引用图片链接,从而得到一个完整的 Markdown 图片格式。

3. 指定回复

指定回复可以直接输出传入的内容到客户端,因此可以直接输出加工好的 Markdown 图片格式即可。

编排代码

{{% details title="编排配置" closed="true" %}}

{
  "nodes": [
    {
      "nodeId": "userGuide",
      "name": "系统配置",
      "intro": "可以配置应用的系统参数",
      "avatar": "/imgs/workflow/userGuide.png",
      "flowNodeType": "userGuide",
      "position": {
        "x": 531.2422736065552,
        "y": -486.7611729549753
      },
      "inputs": [
        {
          "key": "welcomeText",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "string",
          "label": "core.app.Welcome Text",
          "value": ""
        },
        {
          "key": "variables",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "any",
          "label": "core.app.Chat Variable",
          "value": []
        },
        {
          "key": "questionGuide",
          "valueType": "boolean",
          "renderTypeList": [
            "hidden"
          ],
          "label": "core.app.Question Guide",
          "value": false
        },
        {
          "key": "tts",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "any",
          "label": "",
          "value": {
            "type": "web"
          }
        },
        {
          "key": "whisper",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "any",
          "label": "",
          "value": {
            "open": false,
            "autoSend": false,
            "autoTTSResponse": false
          }
        },
        {
          "key": "scheduleTrigger",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "any",
          "label": "",
          "value": null
        }
      ],
      "outputs": []
    },
    {
      "nodeId": "448745",
      "name": "流程开始",
      "intro": "",
      "avatar": "/imgs/workflow/userChatInput.svg",
      "flowNodeType": "workflowStart",
      "position": {
        "x": 532.1275542407774,
        "y": 46.03775600322817
      },
      "inputs": [
        {
          "key": "userChatInput",
          "renderTypeList": [
            "reference",
            "textarea"
          ],
          "valueType": "string",
          "label": "用户问题",
          "required": true,
          "toolDescription": "用户问题"
        }
      ],
      "outputs": [
        {
          "id": "userChatInput",
          "key": "userChatInput",
          "label": "core.module.input.label.user question",
          "valueType": "string",
          "type": "static"
        }
      ]
    },
    {
      "nodeId": "tMyUnRL5jIrC",
      "name": "HTTP 请求",
      "intro": "可以发出一个 HTTP 请求,实现更为复杂的操作(联网搜索、数据库查询等)",
      "avatar": "/imgs/workflow/http.png",
      "flowNodeType": "httpRequest468",
      "showStatus": true,
      "position": {
        "x": 921.2377506442713,
        "y": -483.94114977914256
      },
      "inputs": [
        {
          "key": "system_addInputParam",
          "renderTypeList": [
            "addInputParam"
          ],
          "valueType": "dynamic",
          "label": "",
          "required": false,
          "description": "core.module.input.description.HTTP Dynamic Input",
          "editField": {
            "key": true,
            "valueType": true
          }
        },
        {
          "key": "prompt",
          "valueType": "string",
          "label": "prompt",
          "renderTypeList": [
            "reference"
          ],
          "description": "",
          "canEdit": true,
          "editField": {
            "key": true,
            "valueType": true
          },
          "value": [
            "448745",
            "userChatInput"
          ]
        },
        {
          "key": "system_httpMethod",
          "renderTypeList": [
            "custom"
          ],
          "valueType": "string",
          "label": "",
          "value": "POST",
          "required": true
        },
        {
          "key": "system_httpReqUrl",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "string",
          "label": "",
          "description": "core.module.input.description.Http Request Url",
          "placeholder": "https://api.ai.com/getInventory",
          "required": false,
          "value": "https://api.openai.com/v1/images/generations"
        },
        {
          "key": "system_httpHeader",
          "renderTypeList": [
            "custom"
          ],
          "valueType": "any",
          "value": [
            {
              "key": "Authorization",
              "type": "string",
              "value": "Bearer sk-zsfBsxEU3ApSFGYxF4CdB97e9556412588421823371b9f7b"
            }
          ],
          "label": "",
          "description": "core.module.input.description.Http Request Header",
          "placeholder": "core.module.input.description.Http Request Header",
          "required": false
        },
        {
          "key": "system_httpParams",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "any",
          "value": [],
          "label": "",
          "required": false
        },
        {
          "key": "system_httpJsonBody",
          "renderTypeList": [
            "hidden"
          ],
          "valueType": "any",
          "value": "{\n  \"model\": \"dall-e-3\",\n  \"prompt\": \"{{prompt}}\",\n  \"n\": 1,\n  \"size\": \"1024x1024\"\n}",
          "label": "",
          "required": false
        }
      ],
      "outputs": [
        {
          "id": "system_addOutputParam",
          "key": "system_addOutputParam",
          "type": "dynamic",
          "valueType": "dynamic",
          "label": "",
          "editField": {
            "key": true,
            "valueType": true
          }
        },
        {
          "id": "httpRawResponse",
          "key": "httpRawResponse",
          "label": "原始响应",
          "description": "HTTP请求的原始响应。只能接受字符串或JSON类型响应数据。",
          "valueType": "any",
          "type": "static"
        },
        {
          "id": "DeKGGioBwaMf",
          "type": "dynamic",
          "key": "data[0].url",
          "valueType": "string",
          "label": "data[0].url"
        }
      ]
    },
    {
      "nodeId": "CO3POL8svbbi",
      "name": "文本加工",
      "intro": "可对固定或传入的文本进行加工后输出,非字符串类型数据最终会转成字符串类型。",
      "avatar": "/imgs/workflow/textEditor.svg",
      "flowNodeType": "pluginModule",
      "showStatus": false,
      "position": {
        "x": 1417.5940290051137,
        "y": -478.81889618104356
      },
      "inputs": [
        {
          "key": "system_addInputParam",
          "valueType": "dynamic",
          "label": "动态外部数据",
          "renderTypeList": [
            "addInputParam"
          ],
          "required": false,
          "description": "",
          "canEdit": false,
          "value": "",
          "editField": {
            "key": true
          },
          "dynamicParamDefaultValue": {
            "inputType": "reference",
            "valueType": "string",
            "required": true
          }
        },
        {
          "key": "url",
          "valueType": "string",
          "label": "url",
          "renderTypeList": [
            "reference"
          ],
          "required": true,
          "description": "",
          "canEdit": true,
          "editField": {
            "key": true
          },
          "value": [
            "tMyUnRL5jIrC",
            "DeKGGioBwaMf"
          ]
        },
        {
          "key": "文本",
          "valueType": "string",
          "label": "文本",
          "renderTypeList": [
            "textarea"
          ],
          "required": true,
          "description": "",
          "canEdit": false,
          "value": "![]({{url}})",
          "editField": {
            "key": true
          },
          "maxLength": "",
          "dynamicParamDefaultValue": {
            "inputType": "reference",
            "valueType": "string",
            "required": true
          }
        }
      ],
      "outputs": [
        {
          "id": "text",
          "type": "static",
          "key": "text",
          "valueType": "string",
          "label": "text",
          "description": ""
        }
      ],
      "pluginId": "community-textEditor"
    },
    {
      "nodeId": "7mapnCgHfKW6",
      "name": "指定回复",
      "intro": "该模块可以直接回复一段指定的内容。常用于引导、提示。非字符串内容传入时,会转成字符串进行输出。",
      "avatar": "/imgs/workflow/reply.png",
      "flowNodeType": "answerNode",
      "position": {
        "x": 1922.5628399315042,
        "y": -471.67391598231796
      },
      "inputs": [
        {
          "key": "text",
          "renderTypeList": [
            "textarea",
            "reference"
          ],
          "valueType": "string",
          "label": "core.module.input.label.Response content",
          "description": "core.module.input.description.Response content",
          "placeholder": "core.module.input.description.Response content",
          "selectedTypeIndex": 1,
          "value": [
            "CO3POL8svbbi",
            "text"
          ]
        }
      ],
      "outputs": []
    }
  ],
  "edges": [
    {
      "source": "448745",
      "target": "tMyUnRL5jIrC",
      "sourceHandle": "448745-source-right",
      "targetHandle": "tMyUnRL5jIrC-target-left"
    },
    {
      "source": "tMyUnRL5jIrC",
      "target": "CO3POL8svbbi",
      "sourceHandle": "tMyUnRL5jIrC-source-right",
      "targetHandle": "CO3POL8svbbi-target-left"
    },
    {
      "source": "CO3POL8svbbi",
      "target": "7mapnCgHfKW6",
      "sourceHandle": "CO3POL8svbbi-source-right",
      "targetHandle": "7mapnCgHfKW6-target-left"
    }
  ]
}

{{% /details %}}