旁门左道：借助 HttpClientHandler 拦截请求，体验 Semantic Kernel 插件

前天尝试通过 one-api + dashscope(阿里云灵积) + qwen(通义千问)运行 Semantic Kernel 插件（Plugin） ，结果尝试失败，详见前天的博文。

今天换一种方式尝试，选择了一个旁门左道走走看，看能不能在不使用大模型的情况下让 Semantic Kernel 插件运行起来，这个旁门左道就是从 Stephen Toub 那偷学到的一招 ------ 借助 DelegatingHandler(new HttpClientHandler()) 拦截 HttpClient 请求，直接以模拟数据进行响应。

先创建一个 .NET 控制台项目

dotnet new console
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.Extensions.Http

参照 Semantic Kernel 源码中的示例代码创建一个非常简单的插件 LightPlugin

public class LightPlugin
{
    public bool IsOn { get; set; } = false;

    [KernelFunction]
    [Description("帮看一下灯是开是关")]
    public string GetState() => IsOn ? "on" : "off";

    [KernelFunction]
    [Description("开灯或者关灯")]
    public string ChangeState(bool newState)
    {
        IsOn = newState;
        var state = GetState();
        Console.WriteLine(state == "on" ? $"[开灯啦]" : "[关灯咯]");
        return state;
    }
}

接着创建旁门左道 BackdoorHandler，先实现一个最简单的功能，打印 HttpClient 请求内容

public class BypassHandler() : DelegatingHandler(new HttpClientHandler())
{
    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        Console.WriteLine(await request.Content!.ReadAsStringAsync());
        // return await base.SendAsync(request, cancellationToken);
        return new HttpResponseMessage(HttpStatusCode.OK);
    }
}

然后携 LightPlugin 与 BypassHandler 创建 Semantic Kernel 的 Kernel

var builder = Kernel.CreateBuilder();
builder.Services.AddOpenAIChatCompletion("qwen-max", "sk-xxxxxx");
builder.Services.ConfigureHttpClientDefaults(b =>
    b.ConfigurePrimaryHttpMessageHandler(() => new BypassHandler()));
builder.Plugins.AddFromType<LightPlugin>();
Kernel kernel = builder.Build();

再然后，发送携带 prompt 的请求并获取响应内容

var history = new ChatHistory();
history.AddUserMessage("请开灯");
Console.WriteLine("User > " + history[0].Content);
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

// Enable auto function calling
OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
{
    ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};

var result = await chatCompletionService.GetChatMessageContentAsync(
    history,
    executionSettings: openAIPromptExecutionSettings,
    kernel: kernel);

Console.WriteLine("Assistant > " + result);

运行控制台程序，BypassHandler 就会在控制台输出请求的 json 内容（为了阅读方便对json进行了格式化）：
点击查看 json

{
  "messages": [
    {
      "content": "Assistant is a large language model.",
      "role": "system"
    },
    {
      "content": "\u8BF7\u5F00\u706F",
      "role": "user"
    }
  ],
  "temperature": 1,
  "top_p": 1,
  "n": 1,
  "presence_penalty": 0,
  "frequency_penalty": 0,
  "model": "qwen-max",
  "tools": [
    {
      "function": {
        "name": "LightPlugin-GetState",
        "description": "\u5E2E\u770B\u4E00\u4E0B\u706F\u662F\u5F00\u662F\u5173",
        "parameters": {
          "type": "object",
          "required": [],
          "properties": {}
        }
      },
      "type": "function"
    },
    {
      "function": {
        "name": "LightPlugin-ChangeState",
        "description": "\u5F00\u706F\u6216\u8005\u5173\u706F",
        "parameters": {
          "type": "object",
          "required": [
            "newState"
          ],
          "properties": {
            "newState": {
              "type": "boolean"
            }
          }
        }
      },
      "type": "function"
    }
  ],
  "tool_choice": "auto"
}

为了能反序列化这个 json ，我们需要定义一个类型 ChatCompletionRequest，Sermantic Kernel 中没有现成可以使用的，实现代码如下：
点击查看 ChatCompletionRequest

public class ChatCompletionRequest
{
    [JsonPropertyName("messages")]
    public IReadOnlyList<RequestMessage>? Messages { get; set; }

    [JsonPropertyName("temperature")]
    public double Temperature { get; set; } = 1;

    [JsonPropertyName("top_p")]
    public double TopP { get; set; } = 1;

    [JsonPropertyName("n")]
    public int? N { get; set; } = 1;

    [JsonPropertyName("presence_penalty")]
    public double PresencePenalty { get; set; } = 0;

    [JsonPropertyName("frequency_penalty")]
    public double FrequencyPenalty { get; set; } = 0;

    [JsonPropertyName("model")]
    public required string Model { get; set; }

    [JsonPropertyName("tools")]
    public IReadOnlyList<Tool>? Tools { get; set; }

    [JsonPropertyName("tool_choice")]
    public string? ToolChoice { get; set; }
}

public class RequestMessage
{
    [JsonPropertyName("role")]
    public string? Role { get; set; }

    [JsonPropertyName("name")]
    public string? Name { get; set; }

    [JsonPropertyName("content")]
    public string? Content { get; set; }
}

public class Tool
{
    [JsonPropertyName("function")]
    public FunctionDefinition? Function { get; set; }

    [JsonPropertyName("type")]
    public string? Type { get; set; }
}

public class FunctionDefinition
{
    [JsonPropertyName("name")]
    public string? Name { get; set; }

    [JsonPropertyName("description")]
    public string? Description { get; set; }

    [JsonPropertyName("parameters")]
    public ParameterDefinition Parameters { get; set; }

    public struct ParameterDefinition
    {
        [JsonPropertyName("type")]
        public required string Type { get; set; }

        [JsonPropertyName("description")]
        public string? Description { get; set; }

        [JsonPropertyName("required")]
        public string[]? Required { get; set; }

        [JsonPropertyName("properties")]
        public Dictionary<string, PropertyDefinition>? Properties { get; set; }

        public struct PropertyDefinition
        {
            [JsonPropertyName("type")]
            public required PropertyType Type { get; set; }
        }

        [JsonConverter(typeof(JsonStringEnumConverter))]
        public enum PropertyType
        {
            Number,
            String,
            Boolean
        }
    }
}

有了这个类，我们就可以从请求中获取对应 Plugin 的 function 信息，比如下面的代码：

var function = chatCompletionRequest?.Tools.FirstOrDefault(x => x.Function.Description.Contains("开灯"))?.Function;
var functionName = function.Name;
var parameterName = function.Parameters.Properties.FirstOrDefault(x => x.Value.Type == PropertyType.Boolean).Key;

接下来就是旁门左道的关键，直接在 BypassHandler 中响应 Semantic Kernel 通过 OpenAI.ClientCore 发出的 http 请求。

首先创建用于 json 序列化的类 ChatCompletionResponse：
点击查看 ChatCompletionResponse

public class ChatCompletionResponse
{
    [JsonPropertyName("id")]
    public string? Id { get; set; }

    [JsonPropertyName("object")]
    public string? Object { get; set; }

    [JsonPropertyName("created")]
    public long Created { get; set; }

    [JsonPropertyName("model")]
    public string? Model { get; set; }

    [JsonPropertyName("usage"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public Usage? Usage { get; set; }

    [JsonPropertyName("choices")]
    public List<Choice>? Choices { get; set; }
}

public class Choice
{
    [JsonPropertyName("message")]
    public ResponseMessage? Message { get; set; }

    /// <summary>
    /// The message in this response (when streaming a response).
    /// </summary>
    [JsonPropertyName("delta"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public ResponseMessage? Delta { get; set; }

    [JsonPropertyName("finish_reason")]
    public string? FinishReason { get; set; }

    /// <summary>
    /// The index of this response in the array of choices.
    /// </summary>
    [JsonPropertyName("index")]
    public int Index { get; set; }
}

public class ResponseMessage
{
    [JsonPropertyName("role")]
    public string? Role { get; set; }

    [JsonPropertyName("name"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public string? Name { get; set; }

    [JsonPropertyName("content"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public string? Content { get; set; }

    [JsonPropertyName("tool_calls")]
    public IReadOnlyList<ToolCall>? ToolCalls { get; set; }
}

public class ToolCall
{
    [JsonPropertyName("id")]
    public string? Id { get; set; }

    [JsonPropertyName("function")]
    public FunctionCall? Function { get; set; }

    [JsonPropertyName("type")]
    public string? Type { get; set; }
}

public class Usage
{
    [JsonPropertyName("prompt_tokens")]
    public int PromptTokens { get; set; }

    [JsonPropertyName("completion_tokens")]
    public int CompletionTokens { get; set; }

    [JsonPropertyName("total_tokens")]
    public int TotalTokens { get; set; }
}

public class FunctionCall
{
    [JsonPropertyName("name")]
    public string Name { get; set; } = string.Empty;

    [JsonPropertyName("arguments")]
    public string Arguments { get; set; } = string.Empty;
}

先试试不执行 function calling ，直接以 assistant 角色回复一句话

public class BypassHandler() : DelegatingHandler(new HttpClientHandler())
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var chatCompletion = new ChatCompletionResponse
        {
            Id = Guid.NewGuid().ToString(),
            Model = "fake-mode",
            Object = "chat.completion",
            Created = DateTimeOffset.Now.ToUnixTimeSeconds(),
            Choices =
               [
                   new()
                   {
                       Message = new ResponseMessage
                       {
                           Content = "自己动手，丰衣足食",
                           Role = "assistant"
                       },
                       FinishReason = "stop"
                   }
               ]
        };

        var json = JsonSerializer.Serialize(chatCompletion, GetJsonSerializerOptions());
        return new HttpResponseMessage
        {
            Content = new StringContent(json, Encoding.UTF8, "application/json")
        };
    }
}

运行控制台程序，输出如下：

User > 请开灯
Assistant > 自己动手，丰衣足食

成功响应，到此，旁门左道成功了一半。

接下来在之前创建的 chatCompletion 基础上添加针对 function calling 的 ToolCall 部分。

先准备好 ChangeState(bool newState) 的参数值

Dictionary<string, bool> arguments = new()
{
    { parameterName, true }
};

并将回复内容由 "自己动手，丰衣足食" 改为 "客官，灯已开"

Message = new ResponseMessage
{
    Content = "客官，灯已开",
    Role = "assistant"
}

然后为 chatCompletion 创建 ToolCalls 实例用于响应 function calling

var messages = chatCompletionRequest.Messages;
if (messages.First(x => x.Role == "user").Content.Contains("开灯") == true)
{
    chatCompletion.Choices[0].Message.ToolCalls = new List<ToolCall>()
    {
        new ToolCall
        {
            Id = Guid.NewGuid().ToString(),
            Type = "function",
            Function = new FunctionCall
            {
                Name = function.Name,
                Arguments = JsonSerializer.Serialize(arguments, GetJsonSerializerOptions())
            }
        }
    };
}

运行控制台程序看看效果

User > 请开灯
[开灯啦]
[开灯啦]
[开灯啦]
[开灯啦]
[开灯啦]
Assistant > 客官，灯已开

耶！成功开灯！但是，竟然开了5次，差点把灯给开爆了。

在 BypassHandler 中打印一下请求内容看看哪里出了问题

var json = await request.Content!.ReadAsStringAsync();
Console.WriteLine(json);

原来分别请求/响应了5次，第2次请求开始，json 中 messages 部分多了 tool_calls 与 tool_call_id 内容

{
  "messages": [
    {
      "content": "\u5BA2\u5B98\uFF0C\u706F\u5DF2\u5F00",
      "tool_calls": [
        {
          "function": {
            "name": "LightPlugin-ChangeState",
            "arguments": "{\u0022newState\u0022:true}"
          },
          "type": "function",
          "id": "76f8dead-b5ad-4e6d-b343-7f78d68fac8e"
        }
      ],
      "role": "assistant"
    },
    {
      "content": "on",
      "tool_call_id": "76f8dead-b5ad-4e6d-b343-7f78d68fac8e",
      "role": "tool"
    }
  ]
}

这时恍然大悟，之前 AI assistant 对 function calling 的响应只是让 Plugin 执行对应的 function，assistant 还需要根据执行的结果决定下一下做什么，第2次请求中的 tool_calls 与 tool_call_id 就是为了告诉 assistant 执行的结果，所以，还需要针对这个请求进行专门的响应。

到了旁门左道最后100米冲刺的时刻！

给 RequestMessage 添加 ToolCallId 属性

public class RequestMessage
{
    [JsonPropertyName("role")]
    public string? Role { get; set; }

    [JsonPropertyName("name")]
    public string? Name { get; set; }

    [JsonPropertyName("content")]
    public string? Content { get; set; }

    [JsonPropertyName("tool_call_id")]
    public string? ToolCallId { get; set; }
}

在 BypassHandler 中响应时判断一下 ToolCallId，如果是针对 Plugin 的 function 执行结果的请求，只返回 Message.Content，不进行 function calling 响应

var messages = chatCompletionRequest.Messages;
var toolCallId = "76f8dead- b5ad-4e6d-b343-7f78d68fac8e";
var toolCallIdMessage = messages.FirstOrDefault(x => x.Role == "tool" && x.ToolCallId == toolCallId);

if (toolCallIdMessage != null && toolCallIdMessage.Content == "on")
{
    chatCompletion.Choices[0].Message.Content = "客官，灯已开";
}
else if (messages.First(x => x.Role == "user").Content.Contains("开灯") == true)
{  
    chatCompletion.Choices[0].Message.Content = "";
    //..
}

改进代码完成，到了最后10米冲刺的时刻，再次运行控制台程序

User > 请开灯
[开灯啦]
Assistant > 客官，灯已开

只有一次开灯，冲刺成功，旁门左道走通，用这种方式体验一下 Semantic Kernel Plugin，也别有一番风味。

完整示例代码已上传到 github https://github.com/cnblogs-dudu/sk-plugin-sample-101