How to apply streaming in azure openai dotnet web application?

题意:"如何在 Azure OpenAI 的 .NET Web 应用程序中应用流式处理?"

问题背景:

I want to create a web api backend that stream openai completion responses.

"我想创建一个 Web API 后端,用于流式传输 OpenAI 的完成响应。"

How can I apply the following solution to a web api action in controller?

"如何将以下解决方案应用到控制器中的 Web API 操作?"

cs 复制代码
var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
var chatCompletionsOptions = new ChatCompletionsOptions()
{
    DeploymentName = "gpt-3.5-turbo", // Use DeploymentName for "model" with non-Azure clients
    Messages =
    {
        new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."),
        new ChatRequestUserMessage("Can you help me?"),
        new ChatRequestAssistantMessage("Arrrr! Of course, me hearty! What can I do for ye?"),
        new ChatRequestUserMessage("What's the best way to train a parrot?"),
    }
};

await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
{
    if (chatUpdate.Role.HasValue)
    {
        Console.Write($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");
    }
    if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
    {
        Console.Write(chatUpdate.ContentUpdate);
    }
}

问题解决:

You can simply wrap your code inside the controller

"您可以简单地将代码包裹在控制器内。"

cs 复制代码
using Microsoft.AspNetCore.Mvc;
using OpenAI;
using OpenAI.Chat;
using System.Collections.Generic;
using System.Threading.Tasks;

[ApiController]
[Route("[controller]")]
public class ChatController : ControllerBase
{
    [HttpGet]
    public async Task<ActionResult<List<string>>> GetChatCompletions()
    {
        var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
        var chatCompletionsOptions = new ChatCompletionsOptions()
        {
            DeploymentName = "gpt-3.5-turbo",
            Messages =
            {
                new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."),
                new ChatRequestUserMessage("Can you help me?"),
                new ChatRequestAssistantMessage("Arrrr! Of course, me hearty! What can I do for ye?"),
                new ChatRequestUserMessage("What's the best way to train a parrot?"),
            }
        };

        var responses = new List<string>();

        await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
        {
            if (chatUpdate.Role.HasValue)
            {
                responses.Add($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");
            }
            if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
            {
                responses.Add(chatUpdate.ContentUpdate);
            }
        }

        return Ok(responses);
    }
}

If you don't want to hardcode the message and pass that as a body then you can do something like this

"如果您不想将消息硬编码并作为主体传递,那么您可以这样做"

cs 复制代码
using Microsoft.AspNetCore.Mvc;
using OpenAI;
using OpenAI.Chat;
using System.Collections.Generic;
using System.Threading.Tasks;

[ApiController]
[Route("[controller]")]
public class ChatController : ControllerBase
{
    public class ChatRequest
    {
        public List<string> Messages { get; set; }
    }

    [HttpPost]
    public async Task<ActionResult<List<string>>> PostChatCompletions([FromBody] ChatRequest request)
    {
        var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
        var chatCompletionsOptions = new ChatCompletionsOptions()
        {
            DeploymentName = "gpt-3.5-turbo",
            Messages = new List<ChatRequestMessage>()
        };

        foreach (var message in request.Messages)
        {
            chatCompletionsOptions.Messages.Add(new ChatRequestUserMessage(message));
        }

        var responses = new List<string>();

        await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
        {
            if (chatUpdate.Role.HasValue)
            {
                responses.Add($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");
            }
            if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
            {
                responses.Add(chatUpdate.ContentUpdate);
            }
        }

        return Ok(responses);
    }
}

Remember the above implementation of the API does not support streaming responses. It waits for all the chat completions to be received from the OpenAI API, then sends them all at once to the client.

"请记住,上述 API 实现不支持流式响应。它会等待从 OpenAI API 接收到所有聊天完成后,再将它们一次性发送给客户端。"

Streaming responses to the client as they are received from the OpenAI API would require a different approach. This could be achieved using Server-Sent Events (SSE) or a similar technology, but it's important to note that not all clients and network environments support these technologies.

"将从 OpenAI API 接收到的响应流式传输给客户端需要采用不同的方法。这可以通过使用服务器发送事件 (SSE) 或类似技术来实现,但需要注意的是,并非所有客户端和网络环境都支持这些技术。"

Here's a simplified example of how you could implement this using Server-Sent Events in ASP.NET Core:

"以下是一个使用服务器发送事件 (SSE) 在 ASP.NET Core 中实现此功能的简化示例:"

cs 复制代码
[HttpPost]
public async Task PostChatCompletions([FromBody] ChatRequest request)
{
    var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
    var chatCompletionsOptions = new ChatCompletionsOptions()
    {
        DeploymentName = "gpt-3.5-turbo",
        Messages = new List<ChatRequestMessage>()
    };

    foreach (var message in request.Messages)
    {
        chatCompletionsOptions.Messages.Add(new ChatRequestUserMessage(message));
    }

    Response.Headers.Add("Content-Type", "text/event-stream");

    await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
    {
        if (chatUpdate.Role.HasValue)
        {
            await Response.WriteAsync($"data: {chatUpdate.Role.Value.ToString().ToUpperInvariant()}: \n\n");
        }
        if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
        {
            await Response.WriteAsync($"data: {chatUpdate.ContentUpdate}\n\n");
        }
    }
}
相关推荐
wflynn12 小时前
Google 观察到首个全自动化网络攻击,AI 框架已能独立完成漏洞利用
ai·技术
zdr尽职尽责13 小时前
终端工具凭空消失:Codex + GPT-6 的一次完整排障复盘
数据库·gpt·学习·ai
佛系18915 小时前
从服务端日志精确分离搜索抓取、监控探测与真实流量的实操清单
ai·大模型·seo·geo·结构化数据·indexnow
BruceNeter19 小时前
C# winform卡死问题分析1——线程池子线程创建句柄
c#·.net
一起学安全技术20 小时前
拆大模型 11|大模型面试拷问:12 问逐层拆
人工智能·ai·语言模型·面试·职场和发展·求职招聘
TomEval1 天前
【测AI】第26篇:线上模型监控与告警体系 — 测试不是终点
人工智能·python·功能测试·ai
AI算力2 天前
企业调研——工业边缘计算隐形黑马,腾视科技以“硬件+算法”加速出海落地!
人工智能·ai·边缘计算·ai算力·无人叉车·物理ai·ainas
真上帝的左手2 天前
26. AI-大语言模型-基础
人工智能·ai·架构·llm·大语言模型
何以解忧唯有撸码2 天前
开源】当桌面图标开始打台球:我用 WinForms 做了一桌“真的”台球
c#·winform·自定义控件·gdi
七夜zippoe3 天前
流式输出:让 AI 回答像打字机一样实时返回(SSE 全链路实战)
人工智能·spring·ai·流式输出·ssf