第3讲:代码补全引擎

第2讲我们让 MiniCopilot 学会了"读懂代码"------AST 解析和符号表让它能理解代码的结构和语义。这一讲,我们要实现 MiniCopilot 最核心的功能:代码补全

当开发者在编辑器中敲击键盘时,代码补全引擎需要在毫秒级别预测接下来要输入的代码,并以灰色文字的形式展示在光标后方。


一、代码补全的核心挑战

挑战 说明
低延迟 用户打字时,补全建议必须在 100ms 内出现
高准确率 错误的建议比没有建议更糟糕
上下文敏感 必须理解当前作用域、可用变量、期望类型
增量更新 每次按键都要重新计算,不能每次都全量扫描

补全流程

复制代码
用户按下键盘
    ↓
1. 捕获当前状态(光标位置、附近代码、语言类型)
    ↓
2. 构建上下文(当前作用域的符号、最近打开的关联文件)
    ↓
3. 生成候选(基于 AST 的静态补全 + LLM 的动态补全)
    ↓
4. 排序与过滤(优先级排序、去重、安全检查)
    ↓
5. 返回给 IDE 展示(灰色文字)

二、基于 AST 的静态补全

静态补全是代码补全的第一道防线------它不需要调用 LLM,完全基于代码分析,速度快、确定性高。

2.1 成员补全(. 之后)

当用户输入 obj. 时,我们需要知道 obj 有哪些属性和方法。

复制代码
# engine/completion/member_completion.py
from engine.analyzer.symbol_table import SymbolTableBuilder, SymbolKind
from engine.parser.base import BaseParser

class MemberCompletionProvider:
    """成员补全:在 . 之后提供属性/方法建议"""
    
    def __init__(self, parser: BaseParser, symbol_builder: SymbolTableBuilder):
        self.parser = parser
        self.symbol_builder = symbol_builder
    
    def get_completions(self, code: str, line: int, column: int) -> list[dict]:
        """
        获取成员补全建议
        
        参数:
            code: 完整源代码
            line: 当前行号(从0开始)
            column: 当前列号(从0开始)
        
        返回:
            [{"label": "method_name", "type": "method", "detail": "说明"}, ...]
        """
        # 获取点号前面的表达式
        obj_expr = self._get_object_expression(code, line, column)
        if not obj_expr:
            return []
        
        # 根据表达式推断类型
        obj_type = self._infer_type(obj_expr, code)
        if not obj_type:
            return []
        
        # 根据类型获取成员
        members = self._get_type_members(obj_type)
        return members
    
    def _get_object_expression(self, code: str, line: int, column: int) -> str:
        """获取点号前面的表达式"""
        lines = code.split('\n')
        current_line = lines[line] if line < len(lines) else ""
        
        # 从光标往前找,直到遇到非标识符字符
        before_cursor = current_line[:column]
        
        # 去掉末尾的点
        if before_cursor.endswith('.'):
            before_cursor = before_cursor[:-1]
        
        # 提取最后的标识符
        import re
        match = re.search(r'[\w.]+$', before_cursor)
        if match:
            return match.group()
        return ""
    
    def _infer_type(self, expression: str, code: str) -> str:
        """推断表达式的类型"""
        # 简单情况:直接是变量名
        if '.' not in expression:
            # 在符号表中查找
            scope = self.symbol_builder.build(code)
            symbol = scope.lookup(expression)
            if symbol and symbol.type_hint:
                return symbol.type_hint
            return None
        
        # 复杂情况:链式调用,如 user.name
        parts = expression.split('.')
        # TODO: 需要更复杂的类型推断
        return None
    
    def _get_type_members(self, type_name: str) -> list[dict]:
        """获取类型的成员列表"""
        # 内置类型成员
        builtin_members = {
            "str": [
                {"label": "upper", "type": "method", "detail": "str.upper() -> str"},
                {"label": "lower", "type": "method", "detail": "str.lower() -> str"},
                {"label": "split", "type": "method", "detail": "str.split(sep=None) -> list"},
                {"label": "strip", "type": "method", "detail": "str.strip() -> str"},
                {"label": "replace", "type": "method", "detail": "str.replace(old, new) -> str"},
                {"label": "find", "type": "method", "detail": "str.find(sub) -> int"},
                {"label": "startswith", "type": "method", "detail": "str.startswith(prefix) -> bool"},
                {"label": "endswith", "type": "method", "detail": "str.endswith(suffix) -> bool"},
            ],
            "list": [
                {"label": "append", "type": "method", "detail": "list.append(item) -> None"},
                {"label": "extend", "type": "method", "detail": "list.extend(iterable) -> None"},
                {"label": "insert", "type": "method", "detail": "list.insert(index, item) -> None"},
                {"label": "remove", "type": "method", "detail": "list.remove(item) -> None"},
                {"label": "pop", "type": "method", "detail": "list.pop(index=-1) -> any"},
                {"label": "sort", "type": "method", "detail": "list.sort(key=None, reverse=False) -> None"},
                {"label": "reverse", "type": "method", "detail": "list.reverse() -> None"},
            ],
            "dict": [
                {"label": "keys", "type": "method", "detail": "dict.keys() -> list"},
                {"label": "values", "type": "method", "detail": "dict.values() -> list"},
                {"label": "items", "type": "method", "detail": "dict.items() -> list"},
                {"label": "get", "type": "method", "detail": "dict.get(key, default=None) -> any"},
                {"label": "pop", "type": "method", "detail": "dict.pop(key, default=None) -> any"},
                {"label": "update", "type": "method", "detail": "dict.update(other_dict) -> None"},
            ],
        }
        
        return builtin_members.get(type_name, [])

2.2 关键字补全

当用户开始输入时,提供语言关键字和当前作用域中的符号建议:

复制代码
# engine/completion/keyword_completion.py
class KeywordCompletionProvider:
    """关键字和符号补全"""
    
    # 各语言的关键字
    KEYWORDS = {
        "python": [
            "def", "class", "return", "if", "elif", "else", "for", "while",
            "try", "except", "finally", "with", "as", "import", "from",
            "True", "False", "None", "and", "or", "not", "in", "is",
            "lambda", "yield", "async", "await", "pass", "break", "continue",
        ],
        "javascript": [
            "function", "const", "let", "var", "return", "if", "else",
            "for", "while", "do", "switch", "case", "break", "continue",
            "try", "catch", "finally", "throw", "new", "this", "typeof",
            "true", "false", "null", "undefined", "async", "await",
            "import", "export", "from", "class", "extends", "super",
        ],
    }
    
    def __init__(self, parser, symbol_builder):
        self.parser = parser
        self.symbol_builder = symbol_builder
    
    def get_completions(self, code: str, line: int, column: int, language: str) -> list[dict]:
        """获取补全建议"""
        prefix = self._get_prefix(code, line, column)
        if not prefix:
            return []
        
        suggestions = []
        
        # 1. 语言关键字
        keywords = self.KEYWORDS.get(language, [])
        for kw in keywords:
            if kw.startswith(prefix):
                suggestions.append({
                    "label": kw,
                    "type": "keyword",
                    "detail": f"关键字"
                })
        
        # 2. 当前作用域的符号
        scope = self.symbol_builder.build(code)
        symbols = self._get_all_symbols(scope)
        for name, kind in symbols:
            if name.startswith(prefix):
                suggestions.append({
                    "label": name,
                    "type": kind.value,
                    "detail": f"{kind.value}: {name}"
                })
        
        # 3. 内置函数和常用模块
        builtins = self._get_builtins(language, prefix)
        suggestions.extend(builtins)
        
        # 排序:关键字优先,然后是符号,最后是内置函数
        type_order = {"keyword": 0, "variable": 1, "function": 1, "class": 1, "builtin": 2}
        suggestions.sort(key=lambda x: (type_order.get(x["type"], 99), x["label"]))
        
        return suggestions[:20]  # 最多返回20个
    
    def _get_prefix(self, code: str, line: int, column: int) -> str:
        """获取光标前的单词前缀"""
        lines = code.split('\n')
        current_line = lines[line] if line < len(lines) else ""
        before_cursor = current_line[:column]
        
        # 提取最后一个单词
        import re
        match = re.search(r'[\w_]+$', before_cursor)
        if match:
            return match.group()
        return ""
    
    def _get_all_symbols(self, scope) -> list[tuple[str, SymbolKind]]:
        """递归获取所有作用域中的符号"""
        symbols = []
        for name, sym in scope.symbols.items():
            symbols.append((name, sym.kind))
        for child in scope.children:
            symbols.extend(self._get_all_symbols(child))
        return symbols
    
    def _get_builtins(self, language: str, prefix: str) -> list[dict]:
        """获取内置函数"""
        builtins = {
            "python": [
                "print", "len", "range", "int", "str", "float", "list",
                "dict", "set", "tuple", "type", "isinstance", "hasattr",
                "open", "input", "sorted", "reversed", "enumerate", "zip",
                "map", "filter", "reduce", "any", "all", "sum", "min", "max",
                "abs", "round", "pow", "divmod", "hex", "oct", "bin", "ord", "chr",
            ],
            "javascript": [
                "console.log", "console.error", "JSON.stringify", "JSON.parse",
                "Math.random", "Math.floor", "Math.ceil", "Math.round",
                "Array.from", "Array.isArray", "Object.keys", "Object.values",
                "parseInt", "parseFloat", "setTimeout", "setInterval",
                "fetch", "Promise", "async", "await",
            ],
        }
        
        suggestions = []
        for name in builtins.get(language, []):
            if name.startswith(prefix):
                suggestions.append({
                    "label": name,
                    "type": "builtin",
                    "detail": f"内置函数"
                })
        return suggestions

2.3 完整补全提供者

复制代码
# engine/completion/provider.py
from engine.parser.python_parser import PythonParser
from engine.parser.js_parser import JavaScriptParser
from engine.analyzer.symbol_table import SymbolTableBuilder
from .member_completion import MemberCompletionProvider
from .keyword_completion import KeywordCompletionProvider

class CompletionProvider:
    """补全提供者:整合所有补全策略"""
    
    def __init__(self):
        # 初始化解析器
        self.parsers = {
            "python": PythonParser(),
            "javascript": JavaScriptParser(),
        }
        
        # 初始化各补全策略
        self.strategies = {}
        for lang, parser in self.parsers.items():
            symbol_builder = SymbolTableBuilder(parser)
            self.strategies[lang] = {
                "member": MemberCompletionProvider(parser, symbol_builder),
                "keyword": KeywordCompletionProvider(parser, symbol_builder),
            }
    
    def get_completions(self, code: str, line: int, column: int, 
                       language: str = "python") -> list[dict]:
        """
        获取代码补全建议
        
        参数:
            code: 源代码
            line: 行号(从0开始)
            column: 列号(从0开始)
            language: 编程语言
        
        返回:
            [{"label": "...", "type": "...", "detail": "..."}, ...]
        """
        if language not in self.parsers:
            return []
        
        strategies = self.strategies[language]
        
        # 1. 检查是否是成员补全(在 . 之后)
        before_cursor = self._get_text_before_cursor(code, line, column)
        if before_cursor.endswith('.'):
            return strategies["member"].get_completions(code, line, column)
        
        # 2. 关键字和符号补全
        return strategies["keyword"].get_completions(code, line, column, language)
    
    def _get_text_before_cursor(self, code: str, line: int, column: int) -> str:
        """获取光标前的文本"""
        lines = code.split('\n')
        if line < len(lines):
            return lines[line][:column]
        return ""

三、基于 LLM 的动态补全

静态补全能覆盖很多场景,但对于"猜用户接下来要写什么",LLM 更强大。

3.1 上下文构建

复制代码
# engine/completion/llm_completion.py
import json
import requests
from typing import Optional

class LLMCompletionProvider:
    """基于 LLM 的代码补全"""
    
    def __init__(self, api_key: str, model: str = "deepseek-coder"):
        self.api_key = api_key
        self.model = model
        self.base_url = "https://api.deepseek.com/v1/chat/completions"
    
    def get_completion(self, code: str, line: int, column: int, 
                      language: str = "python", max_new_tokens: int = 50) -> Optional[str]:
        """
        获取 LLM 生成的补全内容
        
        返回:补全的代码文本,或 None
        """
        # 1. 构建上下文
        context = self._build_context(code, line, column, language)
        
        # 2. 调用 LLM
        prompt = self._build_prompt(context, language)
        
        headers = {
            "Authorization": f"Bearer {self.api_key}",
            "Content-Type": "application/json"
        }
        
        payload = {
            "model": self.model,
            "messages": [
                {"role": "system", "content": "You are a code completion engine. Complete the code at the cursor position. Only output the completion code, no explanations."},
                {"role": "user", "content": prompt}
            ],
            "max_tokens": max_new_tokens,
            "temperature": 0.1,  # 低温度,更确定性的输出
            "stop": ["\n\n", "```"]  # 遇到空行或代码块结束符停止
        }
        
        try:
            response = requests.post(self.base_url, headers=headers, json=payload, timeout=5)
            response.raise_for_status()
            completion = response.json()["choices"][0]["message"]["content"]
            
            # 后处理
            completion = self._post_process(completion)
            return completion
            
        except Exception as e:
            print(f"LLM 补全失败: {e}")
            return None
    
    def _build_context(self, code: str, line: int, column: int, language: str) -> dict:
        """构建上下文"""
        lines = code.split('\n')
        
        # 取光标前的代码(前 30 行)
        before_start = max(0, line - 30)
        before_code = '\n'.join(lines[before_start:line])
        before_cursor = lines[line][:column] if line < len(lines) else ""
        
        # 取光标后的代码(后 10 行)
        after_end = min(len(lines), line + 10)
        after_code = '\n'.join(lines[line:after_end])
        after_cursor = lines[line][column:] if line < len(lines) else ""
        
        return {
            "before_cursor": before_code + '\n' + before_cursor,
            "after_cursor": after_cursor + '\n' + after_code,
            "line": line,
            "column": column,
            "language": language
        }
    
    def _build_prompt(self, context: dict, language: str) -> str:
        """构建提示词"""
        return f"""Complete the code at the cursor position (marked by <CURSOR>).

Language: {language}

Code before cursor:

{language}

{context'before_cursor'}<CURSOR>

复制代码
Code after cursor:

{language}

{context'after_cursor'}

复制代码
Only output the code that should replace <CURSOR>. Do not repeat existing code."""
    
    def _post_process(self, completion: str) -> str:
        """后处理补全结果"""
        # 去除首尾空白
        completion = completion.strip()
        
        # 如果补全包含代码块标记,提取代码
        if '```' in completion:
            parts = completion.split('```')
            # 取第二个代码块(索引1)或倒数第二个
            for i, part in enumerate(parts):
                if i % 2 == 1 and not part.startswith('\n'):
                    completion = part.split('\n', 1)[-1] if '\n' in part else part
                    break
        
        return completion

3.2 混合补全策略

复制代码
# engine/completion/hybrid.py
import time
from .provider import CompletionProvider
from .llm_completion import LLMCompletionProvider

class HybridCompletionEngine:
    """
    混合补全引擎
    
    策略:
    1. 先尝试静态补全(< 5ms)
    2. 如果静态补全结果不够好,启动 LLM 补全(~500ms)
    3. 合并结果,去重排序
    """
    
    def __init__(self, api_key: str):
        self.static_provider = CompletionProvider()
        self.llm_provider = LLMCompletionProvider(api_key)
    
    def get_completions(self, code: str, line: int, column: int,
                       language: str = "python") -> dict:
        """
        获取补全建议
        
        返回:
            {
                "static": [...],  # 静态补全结果(即时)
                "llm": "...",     # LLM 补全结果(可能为空)
                "timing": {...}   # 耗时统计
            }
        """
        timing = {}
        
        # 1. 静态补全(同步,快速)
        start = time.time()
        static_results = self.static_provider.get_completions(code, line, column, language)
        timing["static_ms"] = round((time.time() - start) * 1000, 1)
        
        # 2. LLM 补全(异步,慢)
        start = time.time()
        llm_result = self.llm_provider.get_completion(code, line, column, language)
        timing["llm_ms"] = round((time.time() - start) * 1000, 1)
        
        return {
            "static": static_results,
            "llm": llm_result,
            "timing": timing
        }

四、补全排序与过滤

复制代码
# engine/completion/ranker.py
class CompletionRanker:
    """补全结果排序器"""
    
    def __init__(self):
        # 类型优先级(数字越小越优先)
        self.type_priority = {
            "keyword": 0,
            "local_variable": 1,
            "parameter": 2,
            "method": 3,
            "function": 4,
            "class": 5,
            "builtin": 6,
            "member": 7,
        }
    
    def rank(self, completions: list[dict], prefix: str) -> list[dict]:
        """
        对补全结果排序
        
        排序规则:
        1. 精确匹配优先
        2. 类型优先级高的优先
        3. 短名称优先
        4. 字母序
        """
        scored = []
        for comp in completions:
            score = self._calculate_score(comp, prefix)
            scored.append((score, comp))
        
        # 按分数降序排列
        scored.sort(key=lambda x: (-x[0], x[1]["label"]))
        
        return [comp for _, comp in scored]
    
    def _calculate_score(self, completion: dict, prefix: str) -> float:
        """计算补全项的得分"""
        label = completion["label"]
        score = 0.0
        
        # 1. 精确匹配加分
        if label == prefix:
            score += 100
        elif label.startswith(prefix):
            score += 50
        elif prefix in label:
            score += 20
        
        # 2. 驼峰匹配加分
        if self._camel_case_match(label, prefix):
            score += 30
        
        # 3. 类型优先级
        type_priority = self.type_priority.get(completion.get("type", ""), 50)
        score += (100 - type_priority)  # 优先级高的分数更高
        
        # 4. 短名称加分(更简洁)
        score += max(0, 20 - len(label))
        
        return score
    
    def _camel_case_match(self, label: str, prefix: str) -> bool:
        """检查是否驼峰匹配(如 "GT" 匹配 "getTotal")"""
        import re
        # 提取大写字母
        upper_chars = ''.join(re.findall(r'[A-Z]', label))
        return upper_chars.startswith(prefix.upper())
    
    def filter_duplicates(self, completions: list[dict]) -> list[dict]:
        """去重"""
        seen = set()
        unique = []
        for comp in completions:
            key = (comp["label"], comp.get("type", ""))
            if key not in seen:
                seen.add(key)
                unique.append(comp)
        return unique

五、完整测试

复制代码
# test_completion.py
from engine.completion.hybrid import HybridCompletionEngine
from engine.completion.ranker import CompletionRanker

# 初始化(需要 API Key)
api_key = "your-api-key"
engine = HybridCompletionEngine(api_key)
ranker = CompletionRanker()

# 测试代码
code = """
def calculate_total(prices, tax_rate):
    total = 0
    for price in prices:
        total += price
    return total * (1 + tax_rate)

def main():
    items = [100, 200, 300]
    rate = 0.1
    
    # 光标在这里(第9行,第4列)
    result = 
"""

# 测试1:静态补全(关键字)
print("=== 测试1:关键字补全 ===")
result = engine.get_completions(code, line=9, column=11, language="python")
print(f"静态补全 ({result['timing']['static_ms']}ms):")
for comp in result["static"][:5]:
    print(f"  [{comp['type']}] {comp['label']}")

# 测试2:成员补全
print("\n=== 测试2:成员补全 ===")
code_with_dot = code.replace("result = ", "result = items.")
result = engine.get_completions(code_with_dot, line=9, column=18, language="python")
print(f"静态补全 ({result['timing']['static_ms']}ms):")
for comp in result["static"][:5]:
    print(f"  [{comp['type']}] {comp['label']}")

# 测试3:LLM 补全
print("\n=== 测试3:LLM 补全 ===")
if result.get("llm"):
    print(f"LLM 补全 ({result['timing']['llm']}ms):")
    print(f"  {result['llm'][:100]}...")

# 测试4:排序
print("\n=== 测试4:排序测试 ===")
completions = [
    {"label": "calculate_total", "type": "function"},
    {"label": "result", "type": "variable"},
    {"label": "range", "type": "builtin"},
    {"label": "return", "type": "keyword"},
    {"label": "rate", "type": "variable"},
]
ranked = ranker.rank(completions, prefix="r")
print("排序结果:")
for comp in ranked:
    print(f"  [{comp['type']}] {comp['label']}")

六、性能优化

6.1 增量解析

不要每次都全量解析代码,只解析变化的部分:

复制代码
class IncrementalParser:
    """增量解析器"""
    
    def __init__(self, parser):
        self.parser = parser
        self.last_tree = None
        self.last_code = ""
    
    def parse(self, code: str):
        """增量解析"""
        if not self.last_code:
            # 首次解析
            self.last_tree = self.parser.parse(code)
            self.last_code = code
            return self.last_tree
        
        # 计算差异
        import difflib
        diff = list(difflib.unified_diff(
            self.last_code.splitlines(keepends=True),
            code.splitlines(keepends=True)
        ))
        
        # 如果改动很小,只重新解析受影响的区域
        if len(diff) < 10:
            # TODO: 实现局部重解析
            pass
        
        # 改动较大,全量解析
        self.last_tree = self.parser.parse(code)
        self.last_code = code
        return self.last_tree

6.2 缓存

复制代码
import hashlib
from functools import lru_cache

class CompletionCache:
    """补全结果缓存"""
    
    def __init__(self, max_size=100):
        self.cache = {}
        self.max_size = max_size
    
    def _make_key(self, code: str, line: int, column: int, language: str) -> str:
        content = f"{code}|{line}|{column}|{language}"
        return hashlib.md5(content.encode()).hexdigest()
    
    def get(self, code: str, line: int, column: int, language: str):
        key = self._make_key(code, line, column, language)
        return self.cache.get(key)
    
    def set(self, code: str, line: int, column: int, language: str, result):
        key = self._make_key(code, line, column, language)
        
        if len(self.cache) >= self.max_size:
            # 移除最早的缓存
            oldest = next(iter(self.cache))
            del self.cache[oldest]
        
        self.cache[key] = result

七、常见错误 & 排坑指南

  1. 补全建议太多

    • 问题:一次性返回几十个建议,用户难以选择

    • 解决:限制最多 10-20 个,用排序算法精选

  2. LLM 补全太慢

    • 问题:LLM 调用需要 500ms-2s,影响体验

    • 解决:先用静态补全顶住,LLM 结果异步返回

  3. 补全内容包含多余代码

    • 问题:LLM 喜欢生成完整函数而不是简短补全

    • 解决:在 prompt 中明确要求"只输出补全部分",并设置 stop token

  4. 上下文过大导致 token 超限

    • 问题:大型文件的上下文可能超过模型限制

    • 解决:只取光标附近的相关代码,使用滑动窗口


八、课后作业

  1. 实现参数补全:当用户输入函数名和左括号后,提示参数名称和类型。

  2. 添加 Snippet 补全 :预定义一些常用代码模板(如 for 循环、if 语句),输入缩写时展开。

  3. 挑战题:实现"智能导入补全"------当用户使用一个未导入的符号时,自动补全 import 语句。


九、总结

这一讲我们实现了 MiniCopilot 最核心的功能------代码补全:

  • 静态补全:基于 AST 和符号表的成员补全、关键字补全

  • LLM 补全:利用大模型预测接下来的代码

  • 混合策略:静态补全保证速度,LLM 补全提升质量

  • 排序与过滤:让最好的建议排在前面

  • 性能优化:增量解析、缓存

现在,MiniCopilot 已经能在开发者敲代码时提供实时的补全建议了。

下一讲,我们将实现另一个核心能力------自然语言转代码,让用户用中文描述需求,自动生成代码。


🧰 开发之余,处理 Base64、JWT 解析、JSON 格式化、Crontab 计算、PDF 合并压缩这些碎片需求,我常用一个纯前端本地工具箱:zz365.top (子页 PDF 大师:PDF 大师 - zz365工具箱)。所有计算在浏览器完成,文件不上传服务器,关页即清。免费、无登录、无广告,适合开发者当常驻标签页。

相关推荐
小小龙学IT1 小时前
Day 26-27 项目实战:从零构建一个高并发聊天室(epoll + 线程池)
linux·服务器·c语言·开发语言·网络
想会飞的蒲公英1 小时前
PyTorch 学习率实战:从零理解衰减策略与调度器
人工智能·pytorch·python·深度学习·机器学习
布值倒区什么name1 小时前
python文件IO学习
python
我就是妖怪1 小时前
【无标题】
开发语言
丁引2 小时前
《数据清洗的艺术:如何用20行核心逻辑优雅地删除无标签图片》
前端·数据库·python
梦想的旅途22 小时前
企业微信API二次开发:外部群模块功能清单与全场景对接
java·python·企业微信
0566462 小时前
Python高级——迭代器
开发语言·python·学习
初级代码游戏2 小时前
iOS开发 Swift 速记4:函数与闭包
开发语言·ios·swift
进击切图仔2 小时前
在 windows 和 ubuntu 中配置 host 文件
linux·windows·ubuntu