[Loop Engineering在MAF中的实现-03]Agent循环调用流程详解

在Loop Engineering在MAF中的实现-03:Agent循环调用在LoopAgent中的实现中，我们已经对LoopAgent在默认情况下循环调用Agent的流程进行了简单的简单介绍。我们知道，循环调用Agent的总体流程如下：

• 传入用户指定的输入消息和AgentSession调用内部的Agent对象；
• 遍历注册的LoopEvaluator：
  • 如果所有的LoopEvaluator返回的评估结果都表明可以终止循环，那么LoopAgent将会在进行后续的Agent调用；
  • 第一个决定循环继续提供的返回文本或者消息列表将会注入对话历史，作为后续Agent调用的输入。

但是循环的很多行为是可以通过LoopAgentOptions进行控制的。在这篇文章中，我们将利用前面提供的演示实例来系统介绍完整的循环迭代流程。

1. 模拟LLM调用和定义LoopEvaluator

我们定义如下这个实现了IChatClient接口的FakeChatClient类型来模拟针对LLM的调用。如代码所示，我们只实现了基于非流式阻塞式的GetResponseAsync方法。除了返回一个包含单个Assistant消息的响应之外，我们会将输入和输出消息列表打印出来。

class FakeChatClient : IChatClient
{
    public void Dispose() { }
    public Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages, 
        ChatOptions? options = null, 
        CancellationToken cancellationToken = default)
    {
        var message = ChatMessageGenerator.GenerateMessage("Response message from LLM.", ChatRole.Assistant);
        Console.WriteLine($"""
            {new string('-', 30)}LLM{new string('-', 30)}
            Incoming messages:
                {messages.AsString()}
            Outgoing response:
                {message.AsString()}

            """);
        return Task.FromResult(new ChatResponse(message));
    }

    public object? GetService(Type serviceType, object? serviceKey = null) => null;
    public IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
        IEnumerable<ChatMessage> messages, 
        ChatOptions? options = null, 
        CancellationToken cancellationToken = default)
    => throw new NotImplementedException();
}

GetResponseAsync方法中用来生成消息的静态类型ChatMessageGenerator定义如下。GenerateMessage方法根据指定的角色和内容生成对应的消息，为了利于跟踪，我们会在内容前面添加一个自增整数作为前缀。两个AsString扩展方法将指定的ChatMessage和ChatMessage列表转换成字符串。如果设置了AuthorName，我们会将它包含在输出的文本中。

internal static class ChatMessageGenerator
{
    private static int _counter = 0;
    public static ChatMessage GenerateMessage(string content, ChatRole role)
    {
        return new ChatMessage(
            role: role,
            content: $"{Interlocked.Increment(ref _counter)}-{content}"
        );
    }

    public static string AsString(this ChatMessage message)
    => string.IsNullOrEmpty(message.AuthorName)
        ? $"{message}"
        : $"[{message.AuthorName}]{message}";

    public static string AsString(this IEnumerable< ChatMessage> messages)
    => string.Join(Environment.NewLine + new string(' ', 4), messages.Select(AsString)) ;
}

如下这个继承自LoopEvaluator的AlwayContinueEvaluator类型是我们定义的评估器。顾名思义，AlwayContinueEvaluator总是返回一个ShouldReinvoke属性为true的LoopEvaluation，并输出一段包含当前迭代次数的反馈文本。

class AlwayContinueEvaluator : LoopEvaluator
{
    public override ValueTask<LoopEvaluation> EvaluateAsync(LoopContext context, CancellationToken cancellationToken = default)
    {
        var agent = (ChatClientAgent)context.Agent;
        var messagesFromSession = context.Session is null
            ? []
            : ((InMemoryChatHistoryProvider)agent.ChatHistoryProvider!).GetMessages(context.Session);

        Console.WriteLine($"""
            {new string('-', 30)}Evaluator{new string('-', 30)}
            Iteration: {context.Iteration}
            Initial messages:
                {context.InitialMessages.AsString()}
            Last response:
                {context.LastResponse.Messages.AsString()}
            Messages from session:
                {messagesFromSession.AsString()}
            Previous feedback:
                {string.Join("\n    ", context.Feedback)}
            """);
        var feedback = $"Feedback for iteration {context.Iteration}";
        Console.WriteLine($"Feedback:{feedback}\n");
        return ValueTask.FromResult(LoopEvaluation.Continue(feedback));
    }
}

在返回生成的LoopEvaluation之前，我们会输出：

• 当前迭代次数；
• 初始请求列表；
• 本次迭代的响应消息列表；
• 从Session中提取的消息列表；
• 之前迭代生成的反馈列表；
• 本次反馈文本。

2. 默认的循环流程

接下来我们分两种情况来演示采用默认配置（只设置MaxIterations配置选项）下针对Agent的循环调用流程，一种是只提供消息列表，另一种则是提供Session。

2.1 由LoopAgent创建Session

由于LoopAgent总是在Session中对Agent实施循环调用。在默认情况下，如果用户调用LoopAgent时不提供Session，那么LoopAgent会帮助我们创建Session。在如下的演示程序中，我们针对FakeChatClient创建的AIAgent和AlwayContinueEvaluator创建了LoopAgent对象，并通过指定的LoopAgentOptions将最大迭代次数设置为3。我们最后创建了一个包含三个消息的列表作为输入调用了LoopAgent的RunAsync方法。

using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

var loopAgent = new LoopAgent(
    new FakeChatClient().AsAIAgent(), 
    new AlwayContinueEvaluator(),
    new LoopAgentOptions {  MaxIterations =3});

await loopAgent.RunAsync([
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ]);

输出：

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Outgoing response:
    4-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 1
Initial messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Last response:
    4-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
Previous feedback:

Feedback:Feedback for iteration 1

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    Feedback for iteration 1
Outgoing response:
    5-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 2
Initial messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Last response:
    5-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    Feedback for iteration 1
    5-Response message from LLM.
Previous feedback:
    Feedback for iteration 1
Feedback:Feedback for iteration 2

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    Feedback for iteration 1
    5-Response message from LLM.
    Feedback for iteration 2
Outgoing response:
    6-Response message from LLM.

由于最大迭代次数被设置为3，所以整个循环经历三次Agent调用 和两次评估 。上面介绍的默认循环流程体现输出中，LoopAgent自动创建Session也可以从AlwayContinueEvaluator的输出中得到印证。也就是说，虽然LoopEvaluator大部分只需要验证最后一次生成的响应内容与原始请求想匹配，如果某些验证需要使用到完整的对话历史，它可以使用Session。

2.2 用户提供Session

我们对上面演示的程序进行如下的改动：我们从作为内部Agent的ChatClientAgent中提取出ChatHistoryProvider，这是一个InMemoryChatHistoryProvider对象。我们调用它的SetMessages方法将作为输入的三个消息写入创建的Session中，并将它作为参数调用LoopAgent的RunAsync方法。

using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

var innerAgent = new FakeChatClient().AsAIAgent();
var loopAgent = new LoopAgent(
    innerAgent,
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3 });

var session = await innerAgent.CreateSessionAsync();
List<ChatMessage> input = [
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ];
((InMemoryChatHistoryProvider)innerAgent.ChatHistoryProvider!).SetMessages(session, input);

await loopAgent.RunAsync(session);

输出：

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Outgoing response:
    4-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 1
Initial messages:

Last response:
    4-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
Previous feedback:

Feedback:Feedback for iteration 1

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    Feedback for iteration 1
Outgoing response:
    5-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 2
Initial messages:

Last response:
    5-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    Feedback for iteration 1
    5-Response message from LLM.
Previous feedback:
    Feedback for iteration 1
Feedback:Feedback for iteration 2

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    Feedback for iteration 1
    5-Response message from LLM.
    Feedback for iteration 2
Outgoing response:
    6-Response message from LLM.

对于原来的输出可以发现：LLM的输入和输出没有变换，由于原始调用中没有提供消息列表，所以在进行评估的时候，循环上下文LoopContext的InitialMessages为空。

3. 为循环注入的消息打一个标签

LoopAgentOptions的OnBehalfOfAuthorName属性会给循环（LoopEvaluator）自动注入的消息打上作者标记，区分哪些消息是循环自动生成的，哪些是用户真实输入。在如下的演示程序中，我们在指定的LoopAgentOptions配置选项中将OnBehalfOfAuthorName属性设置为LoopAgent。

using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

var loopAgent = new LoopAgent(
    new FakeChatClient().AsAIAgent(),
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3, OnBehalfOfAuthorName ="LoopAgent" });

await loopAgent.RunAsync([
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ]);

输出：

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Outgoing response:
    4-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 1
Initial messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Last response:
    4-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
Previous feedback:

Feedback:Feedback for iteration 1

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    [LoopAgent]Feedback for iteration 1
Outgoing response:
    5-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 2
Initial messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Last response:
    5-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    [LoopAgent]Feedback for iteration 1
    5-Response message from LLM.
Previous feedback:
    Feedback for iteration 1
Feedback:Feedback for iteration 2

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
    [LoopAgent]Feedback for iteration 1
    5-Response message from LLM.
    [LoopAgent]Feedback for iteration 2
Outgoing response:
    6-Response message from LLM.

从输出可以看出，不论是LLM还是Evaluator输出的消息，通过评估反馈消息生成的消息的AuthorName都被设置为LoopAgent。

4. 为每次迭代刷新上下文

由于Session的存在，每次迭代提供给LLM的都是完整的对话历史，意味着前面的迭代的输出和反馈会影响后续迭代。在大部分情况下，这是一种正向的影响，但是在一些情况下并非如此。如果某个迭代产生了错误或无关的回复，默认模式下它会被追加到历史里，影响后续迭代。此时我们可以将LoopAgentOptions的FreshContextPerIteration属性设置成true，那么每次迭代都会刷新上下文，重新从原始输入消息 + 聚合的反馈开始。

4.1 LoopAgent创建Session

如果用户在调用LoopAgent时没有指定Sesssion，LoopAgent通过为每次迭代创建新的Session来达到刷新上下文的目的。在如下的演示程序中，我们将LoopAgentOptions的FreshContextPerIteration属性设置为true。

var loopAgent = new LoopAgent(
    new FakeChatClient().AsAIAgent(),
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3, FreshContextPerIteration = true});

await loopAgent.RunAsync([
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ]);

输出：

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Outgoing response:
    4-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 1
Initial messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Last response:
    4-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
Previous feedback:

Feedback:Feedback for iteration 1

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    ## Feedback

- Feedback for iteration 1
Outgoing response:
    5-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 2
Initial messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Last response:
    5-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    ## Feedback

- Feedback for iteration 1
    5-Response message from LLM.
Previous feedback:
    Feedback for iteration 1
Feedback:Feedback for iteration 2

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    ## Feedback

- Feedback for iteration 1
- Feedback for iteration 2
Outgoing response:
    6-Response message from LLM.

可以看出，对于非首次LLM调用，它们输入的消息都是四条 ，其中三条是原始的输入，最后一条是前面评估反馈的聚合。这意味着前面迭代由LLM生成响应不会作为后续调用的上下文。

4.2 由用户提供Session

如果用户在调用LoopAgent的时候提供了Session，LoopAgent会通过对其序列化的方式生成一个快照，并在每次迭代的时候根据此快照生成Session。在如下的演示程序中，我们将LoopAgentOptions的FreshContextPerIteration属性设置为true，并传入一个包含三条消息的Session。

var innerAgent = new FakeChatClient().AsAIAgent();
var loopAgent = new LoopAgent(
    innerAgent,
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3, FreshContextPerIteration = true });

var session = await innerAgent.CreateSessionAsync();
List<ChatMessage> input = [
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ];
((InMemoryChatHistoryProvider)innerAgent.ChatHistoryProvider!).SetMessages(session, input);

await loopAgent.RunAsync(session);

输出：

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
Outgoing response:
    4-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 1
Initial messages:

Last response:
    4-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    4-Response message from LLM.
Previous feedback:

Feedback:Feedback for iteration 1

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    ## Feedback

- Feedback for iteration 1
Outgoing response:
    5-Response message from LLM.

------------------------------Evaluator------------------------------
Iteration: 2
Initial messages:

Last response:
    5-Response message from LLM.
Messages from session:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    ## Feedback

- Feedback for iteration 1
    5-Response message from LLM.
Previous feedback:
    Feedback for iteration 1
Feedback:Feedback for iteration 2

------------------------------LLM------------------------------
Incoming messages:
    1-Original message from user.
    2-Original response from LLM.
    3-Original message from user.
    ## Feedback

- Feedback for iteration 1
- Feedback for iteration 2
Outgoing response:
    6-Response message from LLM.

5. 屏蔽评估反馈

上面介绍的都是针对循环过程的控制，接下来我们介绍针对最终结果的控制。在默认情况下，LoopEvaluator生成的反馈文本和消息列表会添加到响应的消息列表中（反馈文本会转换成角色为User的消息）。如果只需要LLM响应的内容作为最终的结果，我可以通过将LoopAgentOptions的ExcludeOnBehalfOfMessages属性设置为true来实现。在如下的演示程序中，我们创建了两个LoopAgent对象，其ExcludeOnBehalfOfMessages配置选项分别设置为false和true。在进行了相同的调用后，我们输出响应的消息列表。

using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

List<ChatMessage> input = [
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ];

var response  = await  new LoopAgent(
    new FakeChatClient().AsAIAgent(),
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3, ExcludeOnBehalfOfMessages = false })
    .RunAsync(input);
Console.WriteLine($"""
    ExcludeOnBehalfOfMessages = false:
        {response.Messages.AsString()}

    """);

response = await new LoopAgent(
    new FakeChatClient().AsAIAgent(),
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3, ExcludeOnBehalfOfMessages = true })
    .RunAsync(input);
Console.WriteLine($"""
    ExcludeOnBehalfOfMessages = true:
        {response.Messages.AsString()}

    """);

输出：

ExcludeOnBehalfOfMessages = false:
    4-Response message from LLM.
    Feedback for iteration 1
    5-Response message from LLM.
    Feedback for iteration 2
    6-Response message from LLM.

ExcludeOnBehalfOfMessages = true:
    7-Response message from LLM.
    8-Response message from LLM.
    9-Response message from LLM.

6. 返回最后一次迭代的结果

如果只需要返回最后一次迭代调用Agent的结果，可以按照如下的形式将LoopAgentOptions的NonStreamingReturnsLastResponseOnly属性设置为true。

using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

List<ChatMessage> input = [
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ChatMessageGenerator.GenerateMessage("Original response from LLM.", ChatRole.Assistant),
    ChatMessageGenerator.GenerateMessage("Original message from user.", ChatRole.User),
    ];

var response  = await  new LoopAgent(
    new FakeChatClient().AsAIAgent(),
    new AlwayContinueEvaluator(),
    new LoopAgentOptions { MaxIterations = 3, NonStreamingReturnsLastResponseOnly = true })
    .RunAsync(input);
Console.WriteLine(response);

输出：

6-Response message from LLM.