众所周知，LLM的函数function-calling能力很强悍，解决了大模型与实际业务系统的交互问题。其本质就是函数调用。
从openai官网摘图：
简而言之：
-
LLM起到决策的作用，告知业务系统应该调用什么函数，以及入参是什么。
-
业务系统负责实现对应的函数（比如本地实现，或者调用其他系统提供的服务），并且将函数的响应结果再次抛给LLM。
-
LLM根据响应结果，组织自然语言，继续与业务系统进行交互。
在这里，有很多小伙伴会有一个误区：误以为函数调用是有LLM本身执行的。其实，LLM仅仅做决策，而实际的调用是由业务系统完成的。
现阶段，function-calling能力的实现有两种主流方式：
-
LLM本身支持。
-
利用Prompt模板实现，典型如ReAct模板。
在实际的应用过程中，我们还要解决另一个重要问题：
function-calling触发机制是怎样的？也即：何时要使用function-calling能力，何时不应该使用？
这个问题的处理方式，对于整体流程的运行至关重要。
此时，我们可以使用特定Prompt来解决该问题：
You have access to the following tools:
{json.dumps(tools)}
You can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:
{{
"tool": <name of the selected tool>,
"tool_input": <parameters for the selected tool, matching the tool'
s JSON schema>,
"message": <direct response users content>}
该Prompt告知了LLM：如果需要使用function-calling能力，那么就从tools（tools是预定义的functions）中选取一个最匹配的函数；如果不需要，就用自然语言与用户交互，此时与正常的对话流程无异。输出的格式固定为json，方便解析。
由此，我们受到启发：只要LLM基座够强（能够严格遵循Prompt响应诉求），即使LLM本身不支持function-calling，我们也可以自己实现function-calling，脱离对特定LLM的依赖！
拿到function-calling的结果后，若要用自然语言的形式输出结果，还要再调用一次LLM，对结果进行整合。此时可以使用另一个Prompt：
Please generate a natural language description based on the following question and answer.
Question: [Content of the question]
Answer: [Content of the answer]
Generated Description: The result of [key phrase from the question] is [answer].
If necessary, you can polish the description.Only output the Description, with Chinese language.
该Prompt的作用就是告诉LLM，你要根据我的问题和答案，用自然语言重新描述一遍。这里指定了中文输出，可根据实际需要进行调整。
以下是一个可运行的完整Python脚本：
import requests
import json
import random
# 预置函数定义
tools = [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city e.g. Beijing"
},
"unit": {
"type": "string",
"enum": [
"celsius"
]
}
},
"required": [
"location"
]
}
},
{
"name": "calculator",
"description": "计算器",
"parameters": {
"type": "int",
"properties": {
"a": {
"type": "int",
"description": "the first number"
},
"b": {
"type": "int",
"description": "the second number"
}
},
"required": [
"a",
"b"
]
}
}
]
# 获取天气（随机返回，实际使用可以替换为api调用）
def get_current_weather(*args):
# 定义可能的天气状态
weather_conditions = ["sunny", "cloudy", "rainy", "snowy"]
# 定义可能的温度范围
temperature_min = -10 # 最低温度，摄氏度
temperature_max = 35 # 最高温度，摄氏度
# 随机选择一个天气状态
condition = random.choice(weather_conditions)
# 随机生成一个温度
temperature = random.randint(temperature_min, temperature_max)
# 返回一个描述当前天气的字符串
return f"The weather of {args[0].get('location')} is {condition}, and the temperature is {temperature}°C."
def calculator(args):
return sum(value for value in args.values() if isinstance(value, int))
# 函数映射集合
functions = {
"get_current_weather": get_current_weather,
"calculator": calculator,
}
# 驱动整体流程的入口prompt
entrance_prompt = f"""You have access to the following tools:
{json.dumps(tools)}
You can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:
{{
"tool": <name of the selected tool>,
"tool_input": <parameters for the selected tool, matching the tool's JSON schema>,
"message": <direct response users content>
}}"""
# 请以自然语言的形式对结果进行描述
conformity_prompt = f"""
Please generate a natural language description based on the following question and answer.
Question: [Content of the question]
Answer: [Content of the answer]
Generated Description: The result of [key phrase from the question] is [answer].
If necessary, you can polish the description.
Only output the Description, with Chinese language.
"""
def extract_json(s):
stack = 0
start = s.find('{')
if start == -1:
return None
for i in range(start, len(s)):
if s[i] == '{':
stack += 1
elif s[i] == '}':
stack -= 1
if stack == 0:
return s[start:i + 1]
return None
# 结果包装器，type为func表示是函数调用返回的结果，default表示是自然语言结果。对于func返回的结果，会用LLM再次总结
class ResultWrapper:
def __init__(self, type, result):
self.type = type
self.result = result
# 解析LLM返回的结果，如果有json则去解析json
def parse_result(res):
json_str = extract_json(res["message"]["content"])
if json_str is not None:
obj = json.loads(json_str)
if "tool" in obj:
if obj["tool"] in functions:
fun = functions[obj["tool"]]
return ResultWrapper("func", fun(obj["tool_input"]))
else:
return ResultWrapper("default", obj["message"])
else:
return ResultWrapper("default", res["message"]["content"])
else:
return ResultWrapper("default", res["message"]["content"])
def invokeLLM(messages):
url = "${domain}/v1/chat/completions" #需替换域名
model = ""
payload = {
"model": model,
"messages": messages,
}
payload = json.dumps(payload)
headers = {
'Content-Type': 'application/json'
}
print("PAYLOAD: ", payload)
response = requests.request("POST", url, headers=headers, data=payload)
print("RESPONSE: ", response.text)
print("=======================================================================")
resp = json.loads(response.text)
return resp["choices"][0]
if __name__ == '__main__':
while True:
messages = [
{
"role": "system",
"content": entrance_prompt
}
]
user_input = input('Enter a string: ')
messages.append({
"role": "user",
"content": user_input
})
result_wrapper = parse_result(invokeLLM(messages))
if result_wrapper.type == "func":
messages = [
{
"role": "user",
"content": f"{conformity_prompt}\n\nThe question:{user_input}\nThe answer:{result_wrapper.result}"
}
]
print("FINAL RESULT WITH FUNCTION CALL: ", parse_result(invokeLLM(messages)).result)
else: print("FINAL RESULT: ", result_wrapper.result
实验效果：
Enter a string: 你好
PAYLOAD: {"model": "", "messages": [{"role": "system", "content": "You have access to the following tools:\n[{\"name\": \"get_current_weather\", \"description\": \"Get the current weather in a given location\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city e.g. Beijing\"}, \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\"]}}, \"required\": [\"location\"]}}, {\"name\": \"calculator\", \"description\": \"\\u8ba1\\u7b97\\u5668\", \"parameters\": {\"type\": \"int\", \"properties\": {\"a\": {\"type\": \"int\", \"description\": \"the first number\"}, \"b\": {\"type\": \"int\", \"description\": \"the second number\"}}, \"required\": [\"a\", \"b\"]}}]\nYou can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:\n{\n \"tool\": <name of the selected tool>,\n \"tool_input\": <parameters for the selected tool, matching the tool's JSON schema>,\n \"message\": <direct response users content>\n}"}, {"role": "user", "content": "\u4f60\u597d"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\"tool\": null, \"tool_input\": null, \"message\": \"你好，有什么可以帮您的吗？\"}\n```","function_call":null},"finish_reason":"stop"}],"queueTime":0.0020923614501953125,"costTime":0.7685532569885254,"usage":{"prompt_token":244,"completion_token":29,"total_tokens":273}}
=======================================================================
FINAL RESULT: 你好，有什么可以帮您的吗？
Enter a string: 厦门天气如何？
PAYLOAD: {"model": "", "messages": [{"role": "system", "content": "You have access to the following tools:\n[{\"name\": \"get_current_weather\", \"description\": \"Get the current weather in a given location\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city e.g. Beijing\"}, \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\"]}}, \"required\": [\"location\"]}}, {\"name\": \"calculator\", \"description\": \"\\u8ba1\\u7b97\\u5668\", \"parameters\": {\"type\": \"int\", \"properties\": {\"a\": {\"type\": \"int\", \"description\": \"the first number\"}, \"b\": {\"type\": \"int\", \"description\": \"the second number\"}}, \"required\": [\"a\", \"b\"]}}]\nYou can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:\n{\n \"tool\": <name of the selected tool>,\n \"tool_input\": <parameters for the selected tool, matching the tool's JSON schema>,\n \"message\": <direct response users content>\n}"}, {"role": "user", "content": "\u53a6\u95e8\u5929\u6c14\u5982\u4f55\uff1f"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\"tool\": \"get_current_weather\", \"tool_input\": {\"location\": \"Xiamen\", \"unit\": \"celsius\"}, \"message\": \"\"}\n```","function_call":null},"finish_reason":"stop"}],"queueTime":0.0021338462829589844,"costTime":0.9370713233947754,"usage":{"prompt_token":247,"completion_token":36,"total_tokens":283}}
=======================================================================
PAYLOAD: {"model": "", "messages": [{"role": "user", "content": "\nPlease generate a natural language description based on the following question and answer.\nQuestion: [Content of the question]\nAnswer: [Content of the answer]\nGenerated Description: The result of [key phrase from the question] is [answer].\nIf necessary, you can polish the description.\nOnly output the Description, with Chinese language.\n\n\nThe question:\u53a6\u95e8\u5929\u6c14\u5982\u4f55\uff1f\nThe answer:The weather of Xiamen is cloudy, and the temperature is 35\u00b0C."}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"厦门天气情况是：多云，气温35°C。","function_call":null},"finish_reason":"stop"}],"queueTime":0.008246660232543945,"costTime":0.3240656852722168,"usage":{"prompt_token":143,"completion_token":12,"total_tokens":155}}
=======================================================================
FINAL RESULT WITH FUNCTION CALL: 厦门天气情况是：多云，气温35°C。
Enter a string: 383加上135721等于多少？
PAYLOAD: {"model": "", "messages": [{"role": "system", "content": "You have access to the following tools:\n[{\"name\": \"get_current_weather\", \"description\": \"Get the current weather in a given location\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city e.g. Beijing\"}, \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\"]}}, \"required\": [\"location\"]}}, {\"name\": \"calculator\", \"description\": \"\\u8ba1\\u7b97\\u5668\", \"parameters\": {\"type\": \"int\", \"properties\": {\"a\": {\"type\": \"int\", \"description\": \"the first number\"}, \"b\": {\"type\": \"int\", \"description\": \"the second number\"}}, \"required\": [\"a\", \"b\"]}}]\nYou can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:\n{\n \"tool\": <name of the selected tool>,\n \"tool_input\": <parameters for the selected tool, matching the tool's JSON schema>,\n \"message\": <direct response users content>\n}"}, {"role": "user", "content": "383\u52a0\u4e0a135721\u7b49\u4e8e\u591a\u5c11\uff1f"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\"tool\": \"calculator\", \"tool_input\": {\"a\": 383, \"b\": 135721}, \"message\": null}\n```","function_call":null},"finish_reason":"stop"}],"queueTime":0.0021514892578125,"costTime":0.9161381721496582,"usage":{"prompt_token":252,"completion_token":35,"total_tokens":287}}
=======================================================================
PAYLOAD: {"model": "", "messages": [{"role": "user", "content": "\nPlease generate a natural language description based on the following question and answer.\nQuestion: [Content of the question]\nAnswer: [Content of the answer]\nGenerated Description: The result of [key phrase from the question] is [answer].\nIf necessary, you can polish the description.\nOnly output the Description, with Chinese language.\n\n\nThe question:383\u52a0\u4e0a135721\u7b49\u4e8e\u591a\u5c11\uff1f\nThe answer:136104"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"383加上135721等于136104。","function_call":null},"finish_reason":"stop"}],"queueTime":0.0064160823822021484,"costTime":0.28981900215148926,"usage":{"prompt_token":134,"completion_token":11,"total_tokens":145}}
=======================================================================
FINAL RESULT WITH FUNCTION CALL: 383加上135721等于136104。
在这个例子中，预置了两个函数，分别为天气查询和计算器，实验效果中进行了三轮，其中第一次属于未命中函数调用的闲聊场景，后两次分别命中了天气查询和计算器。
在实际的工作中，可能需要预置非常多函数能力，此时可能需要考虑到LLM的输入token限制，必要时需要进行模块划分，将一次LLM决策转化为多次决策，更通用一点的说法就是意图层级识别。