多模态对话（OpenAI Chat Completions 协议）

接口概述

支持图片、音频、文件的视觉理解模型，兼容 OpenAI Chat Completions 格式：可对图片做描述、分析、对比，支持音频转写与理解，支持文件内容提取与问答。多模态与纯文本共用同一个接口，区别只在 messages[].content 的写法。

纯文本对话的完整参数表见「OpenAI 兼容-Chat」页，本页只讲多模态部分。

接口地址

POST https://test.jw-info.com/v1/chat/completions

请求头

与纯文本的区别

多模态场景下 messages[].content 是一个内容块数组，每个元素代表一种内容类型；纯文本场景中 content 是字符串。

// 多模态：content 为数组
"content": [
  { "type": "text", "text": "描述这张图片" },
  { "type": "image_url", "image_url": { "url": "https://example.com/photo.jpg" } }
]

// 纯文本：content 为字符串（仍支持）
"content": "你好，请帮我解答这个问题"

内容块类型

其余请求参数（model、temperature、max_tokens、stream、tools 等）与纯文本一致，见「OpenAI 兼容-Chat」页。

请求示例

1. 图片 URL 方式

curl https://test.jw-info.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-你的密钥" \
  -d '{
    "model": "qwen3.7-plus",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "这张图片里有什么？请详细描述。" },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://example.com/sample.jpg",
              "detail": "high"
            }
          }
        ]
      }
    ]
  }'

from openai import OpenAI

client = OpenAI(api_key="sk-你的密钥", base_url="https://test.jw-info.com/v1")

resp = client.chat.completions.create(
    model="qwen3.7-plus",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "这张图片里有什么？请详细描述。"},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/sample.jpg",
                                  "detail": "high"},
                },
            ],
        }
    ],
)
print(resp.choices[0].message.content)

2. 图片 Base64 方式

图片不便于放到公网时，直接传 data: URL：

curl https://test.jw-info.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-你的密钥" \
  -d '{
    "model": "qwen3.7-plus",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "请分析这张图片中的文字内容。" },
          {
            "type": "image_url",
            "image_url": { "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==" }
          }
        ]
      }
    ]
  }'

import base64


def encode_image(image_path: str) -> str:
    """本地图片 -> data URL（MIME 类型按扩展名给出）"""
    with open(image_path, "rb") as f:
        data = base64.b64encode(f.read()).decode("utf-8")
    return f"data:image/{image_path.rsplit('.', 1)[-1]};base64,{data}"


def describe_image(image_path: str, prompt: str = "描述这张图片",
                   model: str = "qwen3.7-plus") -> str:
    resp = client.chat.completions.create(
        model=model,
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": encode_image(image_path)}},
            ],
        }],
    )
    return resp.choices[0].message.content

Base64 会显著放大请求体，建议只在图片不可公网访问时使用；接口的请求体上限为 64MB。

3. 多图对比

同一条消息里放多个 image_url 内容块即可：

curl https://test.jw-info.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-你的密钥" \
  -d '{
    "model": "qwen3.7-plus",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "这两张图片有什么相同和不同之处？" },
          { "type": "image_url", "image_url": { "url": "https://example.com/image1.jpg" } },
          { "type": "image_url", "image_url": { "url": "https://example.com/image2.jpg" } }
        ]
      }
    ]
  }'

4. 音频输入

音频以 Base64 传入，并声明格式：

curl https://test.jw-info.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-你的密钥" \
  -d '{
    "model": "qwen3.7-plus",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "请将这段录音转写成文字。" },
          {
            "type": "input_audio",
            "input_audio": {
              "data": "UklGRiQAAABXQVZFZm10IBAAAAABAAEARKwAAIhYAQACABAAZGF0YQAAAAA=",
              "format": "wav"
            }
          }
        ]
      }
    ]
  }'

import base64


def transcribe_audio(audio_path: str, prompt: str = "请转写这段音频",
                     model: str = "qwen3.7-plus") -> str:
    with open(audio_path, "rb") as f:
        audio_b64 = base64.b64encode(f.read()).decode("utf-8")
    resp = client.chat.completions.create(
        model=model,
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "input_audio",
                 "input_audio": {"data": audio_b64,
                                 "format": audio_path.rsplit(".", 1)[-1]}},
            ],
        }],
    )
    return resp.choices[0].message.content

5. 文件输入

curl https://test.jw-info.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-你的密钥" \
  -d '{
    "model": "qwen3.7-plus",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "请总结这份文件的主要内容。" },
          {
            "type": "file",
            "file": {
              "file_data": "VGhpcyBpcyBhIHNhbXBsZSBmaWxlIGNvbnRlbnQu",
              "file_name": "report.pdf"
            }
          }
        ]
      }
    ]
  }'

import base64


def analyze_file(file_path: str, prompt: str = "请总结这个文件",
                 model: str = "qwen3.7-plus") -> str:
    with open(file_path, "rb") as f:
        file_b64 = base64.b64encode(f.read()).decode("utf-8")
    resp = client.chat.completions.create(
        model=model,
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "file",
                 "file": {"file_data": file_b64,
                          "file_name": file_path.replace("\\", "/").split("/")[-1]}},
            ],
        }],
    )
    return resp.choices[0].message.content

音频与文件输入的支持度随模型而异，模型不支持时会返回参数错误；遇到就换支持该能力的模型，或改用图片输入。

6. 纯文本（兼容模式）

content 直接写成字符串即可，无需数组形式：

{
  "model": "qwen3.7-plus",
  "messages": [
    { "role": "user", "content": "你好，请介绍一下你自己" }
  ]
}

响应格式

响应结构与纯文本 Chat Completions 相同：

{
  "id": "chatcmpl-xxxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1710000000,
  "model": "qwen3.7-plus",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "这张图片展示了一座位于海边的灯塔，天空晴朗……"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 150, "completion_tokens": 80, "total_tokens": 230 }
}

流式响应

请求体带 stream: true 时按 SSE 返回增量帧，最后以 data: [DONE] 结束：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"这是"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"一张"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"图片"},"finish_reason":null}]}

data: [DONE]

流式响应末尾会返回一帧只带 usage 的统计，无需调用方设置；中途断开时按已产生的用量计费。

错误格式

{
  "error": {
    "message": "Invalid image URL: connection timeout",
    "type": "invalid_request_error",
    "code": "invalid_image_url"
  }
}

状态码与错误处理约定见「错误码」页。

可用模型

以下模型支持多模态（图片 + 音频 + 文件）能力，也都支持纯文本对话：

模型广场上带「视觉理解」标签的模型都支持图片输入；音频与文件输入只有部分模型支持，以实际调用结果为准。模型名与单价以模型广场为准。

计费说明

• 图片、音频、文件都会被折算成输入 token，与文本一起按模型单价计费；输出按输出单价计费。
• 每次调用的费用记入控制台「费用中心」，接口响应本身不返回费用。
• 余额不足会返回 402，需要管理员在控制台手工授信。