How to apply streaming in azure openai dotnet web application?

题意："如何在 Azure OpenAI 的 .NET Web 应用程序中应用流式处理？"

问题背景：

I want to create a web api backend that stream openai completion responses.

"我想创建一个 Web API 后端，用于流式传输 OpenAI 的完成响应。"

How can I apply the following solution to a web api action in controller?

"如何将以下解决方案应用到控制器中的 Web API 操作？"

var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
var chatCompletionsOptions = new ChatCompletionsOptions()
{
    DeploymentName = "gpt-3.5-turbo", // Use DeploymentName for "model" with non-Azure clients
    Messages =
    {
        new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."),
        new ChatRequestUserMessage("Can you help me?"),
        new ChatRequestAssistantMessage("Arrrr! Of course, me hearty! What can I do for ye?"),
        new ChatRequestUserMessage("What's the best way to train a parrot?"),
    }
};

await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
{
    if (chatUpdate.Role.HasValue)
    {
        Console.Write($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");
    }
    if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
    {
        Console.Write(chatUpdate.ContentUpdate);
    }
}

问题解决：

You can simply wrap your code inside the controller

"您可以简单地将代码包裹在控制器内。"

using Microsoft.AspNetCore.Mvc;
using OpenAI;
using OpenAI.Chat;
using System.Collections.Generic;
using System.Threading.Tasks;

[ApiController]
[Route("[controller]")]
public class ChatController : ControllerBase
{
    [HttpGet]
    public async Task<ActionResult<List<string>>> GetChatCompletions()
    {
        var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
        var chatCompletionsOptions = new ChatCompletionsOptions()
        {
            DeploymentName = "gpt-3.5-turbo",
            Messages =
            {
                new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."),
                new ChatRequestUserMessage("Can you help me?"),
                new ChatRequestAssistantMessage("Arrrr! Of course, me hearty! What can I do for ye?"),
                new ChatRequestUserMessage("What's the best way to train a parrot?"),
            }
        };

        var responses = new List<string>();

        await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
        {
            if (chatUpdate.Role.HasValue)
            {
                responses.Add($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");
            }
            if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
            {
                responses.Add(chatUpdate.ContentUpdate);
            }
        }

        return Ok(responses);
    }
}

If you don't want to hardcode the message and pass that as a body then you can do something like this

"如果您不想将消息硬编码并作为主体传递，那么您可以这样做"

using Microsoft.AspNetCore.Mvc;
using OpenAI;
using OpenAI.Chat;
using System.Collections.Generic;
using System.Threading.Tasks;

[ApiController]
[Route("[controller]")]
public class ChatController : ControllerBase
{
    public class ChatRequest
    {
        public List<string> Messages { get; set; }
    }

    [HttpPost]
    public async Task<ActionResult<List<string>>> PostChatCompletions([FromBody] ChatRequest request)
    {
        var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
        var chatCompletionsOptions = new ChatCompletionsOptions()
        {
            DeploymentName = "gpt-3.5-turbo",
            Messages = new List<ChatRequestMessage>()
        };

        foreach (var message in request.Messages)
        {
            chatCompletionsOptions.Messages.Add(new ChatRequestUserMessage(message));
        }

        var responses = new List<string>();

        await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
        {
            if (chatUpdate.Role.HasValue)
            {
                responses.Add($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");
            }
            if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
            {
                responses.Add(chatUpdate.ContentUpdate);
            }
        }

        return Ok(responses);
    }
}

Remember the above implementation of the API does not support streaming responses. It waits for all the chat completions to be received from the OpenAI API, then sends them all at once to the client.

"请记住，上述 API 实现不支持流式响应。它会等待从 OpenAI API 接收到所有聊天完成后，再将它们一次性发送给客户端。"

Streaming responses to the client as they are received from the OpenAI API would require a different approach. This could be achieved using Server-Sent Events (SSE) or a similar technology, but it's important to note that not all clients and network environments support these technologies.

"将从 OpenAI API 接收到的响应流式传输给客户端需要采用不同的方法。这可以通过使用服务器发送事件 (SSE) 或类似技术来实现，但需要注意的是，并非所有客户端和网络环境都支持这些技术。"

Here's a simplified example of how you could implement this using Server-Sent Events in ASP.NET Core:

"以下是一个使用服务器发送事件 (SSE) 在 ASP.NET Core 中实现此功能的简化示例："

[HttpPost]
public async Task PostChatCompletions([FromBody] ChatRequest request)
{
    var client = new OpenAIClient(nonAzureOpenAIApiKey, new OpenAIClientOptions());
    var chatCompletionsOptions = new ChatCompletionsOptions()
    {
        DeploymentName = "gpt-3.5-turbo",
        Messages = new List<ChatRequestMessage>()
    };

    foreach (var message in request.Messages)
    {
        chatCompletionsOptions.Messages.Add(new ChatRequestUserMessage(message));
    }

    Response.Headers.Add("Content-Type", "text/event-stream");

    await foreach (StreamingChatCompletionsUpdate chatUpdate in client.GetChatCompletionsStreaming(chatCompletionsOptions))
    {
        if (chatUpdate.Role.HasValue)
        {
            await Response.WriteAsync($"data: {chatUpdate.Role.Value.ToString().ToUpperInvariant()}: \n\n");
        }
        if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))
        {
            await Response.WriteAsync($"data: {chatUpdate.ContentUpdate}\n\n");
        }
    }
}