OpenAI GPT-3 API error: “This model‘s maximum context length is 4097 tokens“

题意:OpenAI GPT-3 API 错误:"此模型的最大上下文长度为 4097 个 token"

问题背景:

I am making a request to the completions endpoint. My prompt is 1360 tokens, as verified by the Playground and the Tokenizer. I won't show the prompt as it's a little too long for this question.

我正在向补全(completions)端点发送一个请求。我的提示(prompt)包含1360个标记(tokens),这一点已通过(Playground)和标记器(Tokenizer)验证。我不会展示这个提示,因为它对于这个问题来说有点过长。

Here is my request to openai in Nodejs using the openai npm package.

以下是我在 Node.js 中使用 openai npm 包向 OpenAI 发送请求的代码。

python 复制代码
const response = await openai.createCompletion({
  model: 'text-davinci-003',
  prompt,
  max_tokens: 4000,
  temperature: 0.2
})

When testing in the playground my total tokens after response are 1374.

在 Playground 中测试时,我的响应后的总 token 数是 1374。

When submitting my prompt via the completions API I am getting the following error:

当我通过 completions API 提交我的提示时,我收到以下错误:

python 复制代码
error: {
  message: "This model's maximum context length is 4097 tokens, however you requested 5360 tokens (1360 in your prompt; 4000 for the completion). Please reduce your prompt; or completion length.",
  type: 'invalid_request_error',
  param: null,
  code: null
}

If you have been able to solve this one, I'd love to hear how you did it.

如果你已经解决了这个问题,我很想知道你是怎么做到的。

问题解决:

The max_tokens parameter is shared between the prompt and the completion. Tokens from the prompt and the completion all together should not exceed the token limit of a particular OpenAI model.

`max_tokens` 参数在提示和完成之间是共享的。提示和完成的 token 总数不应超过特定 OpenAI 模型的 token 限制。

As stated in the official OpenAI article:

正如 OpenAI 官方文章中所述:

Depending on the model used, requests can use up to 4097 tokens shared between prompt and completion. If your prompt is 4000 tokens, your completion can be 97 tokens at most.

根据所使用的模型,请求可以使用最多 4097 个 token,在提示和完成之间共享。如果你的提示是 4000 个 token,那么你的完成最多可以有 97 个 token。

The limit is currently a technical limitation, but there are often creative ways to solve problems within the limit, e.g. condensing your prompt, breaking the text into smaller pieces, etc.

这个限制目前是技术上的限制,但通常可以通过创造性的方法在限制范围内解决问题,例如:精简提示、将文本拆分成更小的部分等。

Note: For counting tokens before(!) sending an API request, see this answer.

注意:有关在发送 API 请求之前计算 token 的信息,请参阅 [这个回答](https://stackoverflow.com/a/71453182)。

GPT-4 and GPT-4 Turbo models:

LATEST MODEL DESCRIPTION MAX TOKENS TRAINING DATA
gpt-4-1106-preview GPT-4 Turbo The latest GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. This preview model is not yet suited for production traffic. Learn more. 128,000 tokens Up to Apr 2023
gpt-4-vision-preview GPT-4 Turbo with vision Ability to understand images, in addition to all other GPT-4 Turbo capabilties. Returns a maximum of 4,096 output tokens. This is a preview model version and not suited yet for production traffic. Learn more. 128,000 tokens Up to Apr 2023
gpt-4 Currently points to gpt-4-0613. See continuous model upgrades. 8,192 tokens Up to Sep 2021
gpt-4-0613 Snapshot of gpt-4 from June 13th 2023 with improved function calling support. 8,192 tokens Up to Sep 2021
gpt-4-32k Currently points to gpt-4-32k-0613. See continuous model upgrades. 32,768 tokens Up to Sep 2021
gpt-4-32k-0613 Snapshot of gpt-4-32k from June 13th 2023 with improved function calling support. 32,768 tokens Up to Sep 2021
gpt-4-0314 (Legacy) Snapshot of gpt-4 from March 14th 2023 with function calling support. This model version will be deprecated on June 13th 2024. 8,192 tokens Up to Sep 2021
gpt-4-32k-0314 (Legacy) Snapshot of gpt-4-32k from March 14th 2023 with function calling support. This model version will be deprecated on June 13th 2024. 32,768 tokens Up to Sep 2021
LATEST MODEL DESCRIPTION MAX TOKENS TRAINING DATA
gpt-3.5-turbo-1106 Updated GPT 3.5 Turbo The latest GPT-3.5 Turbo model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. Learn more. 16,385 tokens Up to Sep 2021
gpt-3.5-turbo Currently points to gpt-3.5-turbo-0613. Will point to gpt-3.5-turbo-1106 starting Dec 11, 2023. See continuous model upgrades. 4,096 tokens Up to Sep 2021
gpt-3.5-turbo-16k Currently points to gpt-3.5-turbo-0613. Will point to gpt-3.5-turbo-1106 starting Dec 11, 2023. See continuous model upgrades. 16,385 tokens Up to Sep 2021
gpt-3.5-turbo-instruct Similar capabilities as text-davinci-003 but compatible with legacy Completions endpoint and not Chat Completions. 4,096 tokens Up to Sep 2021
gpt-3.5-turbo-0613 (Legacy) Snapshot of gpt-3.5-turbo from June 13th 2023. Will be deprecated on June 13, 2024. 4,096 tokens Up to Sep 2021
gpt-3.5-turbo-16k-0613 (Legacy) Snapshot of gpt-3.5-16k-turbo from June 13th 2023. Will be deprecated on June 13, 2024. 16,385 tokens Up to Sep 2021
gpt-3.5-turbo-0301 (Legacy) Snapshot of gpt-3.5-turbo from March 1st 2023. Will be deprecated on June 13th 2024. 4,096 tokens Up to Sep 2021

GPT-3 models (Legacy):

LATEST MODEL DESCRIPTION MAX TOKENS TRAINING DATA
text-curie-001 Very capable, faster and lower cost than Davinci. 2,049 tokens Up to Oct 2019
text-babbage-001 Capable of straightforward tasks, very fast, and lower cost. 2,049 tokens Up to Oct 2019
text-ada-001 Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost. 2,049 tokens Up to Oct 2019
davinci Most capable GPT-3 model. Can do any task the other models can do, often with higher quality. 2,049 tokens Up to Oct 2019
curie Very capable, but faster and lower cost than Davinci. 2,049 tokens Up to Oct 2019
babbage Capable of straightforward tasks, very fast, and lower cost. 2,049 tokens Up to Oct 2019
ada Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost. 2,049 tokens Up to Oct 2019

GPT base models:

LATEST MODEL DESCRIPTION MAX TOKENS TRAINING DATA
babbage-002 Replacement for the GPT-3 ada and babbage base models. 16,384 tokens Up to Sep 2021
davinci-002 Replacement for the GPT-3 curie and davinci base models. 16,384 tokens Up to Sep 2021
相关推荐
新智元2 小时前
毕业 7 年,身价破亿!清北 AI 天团血洗硅谷,奥特曼被逼分天价股份
人工智能·openai
新智元3 小时前
刚刚,苹果大模型团队负责人叛逃 Meta!华人 AI 巨星 + 1,年薪飙至 9 位数
人工智能·openai
掉鱼的猫3 小时前
Java MCP 实战:构建跨进程与远程的工具服务
java·openai·mcp
海豚调度4 小时前
Linux 基金会报告解读:开源 AI 重塑经济格局,有人失业,有人涨薪!
大数据·人工智能·ai·开源
令狐少侠20115 小时前
ai之RAG本地知识库--基于OCR和文本解析器的新一代RAG引擎:RAGFlow 认识和源码剖析
人工智能·ai
哪吒编程8 小时前
马斯克掀起"帐篷文化",Grok4即将发布,碾压Gemini 2.5 Pro、Claude Opus 4,国内直接使用
openai·grok
小屁妞9 小时前
Spring AI Alibaba智能测试用例生成
ai·测试用例生成·ai生成测试用例
waynaqua9 小时前
FastAPI开发AI应用一:实现连续多轮对话
python·openai
waynaqua9 小时前
FastAPI开发AI应用二:多厂商模型使用指南
python·openai
tonngw11 小时前
DeepSeek 帮助自己的工作
ai