Laya、Kev、NanoJev调用体验

服务器上的三个决策模型

服务器是（Ubuntu，没有显卡，用 CPU 跑）。本机的 8010、8011 已经被别的程序占用，所以这三个服务用新端口，并且只监听服务器自己的 127.0.0.1。

服务	服务器上的地址	权重
Laya	http://127.0.0.1:18110	多语言版，3.22 亿参数
Kev	http://127.0.0.1:18008	Kev-0.8B
NanoJev	http://127.0.0.1:18111	NanoJev 0.6B

健康检查

复制代码

curl -sS http://127.0.0.1:18110/health
curl -sS http://127.0.0.1:18008/v1/models
curl -sS http://127.0.0.1:18111/api/health

Laya

复制代码

curl -sS http://127.0.0.1:18110/v1/systemone \
  -H 'content-type: application/json' \
  -d '{
    "state": {"body": "三月被扣了两次款，请今天退款，不然我们就取消套餐。"},
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "这个工单应该交给哪个部门？",
        "criteria": {
          "billing": "发票、扣款、退款",
          "tech": "故障、bug",
          "other": "其他"
        }
      },
      "urgent": {"type": "noul", "instructions": "是否需要今天处理？"},
      "anger": {
        "type": "score",
        "instructions": "客户有多着急？",
        "criteria": ["不急", "尽快", "立刻"]
      }
    }
  }'

看 answers.department.choice（部门）、answers.urgent.noul（"是"的概率）、answers.anger.score（0 是不急，1 是尽快，2 是立刻）。

bash 复制代码

响应结果：
{
    "model": "laya-rl-agent",
    "answers": {
        "department": {
            "type": "choice",
            "choice": "billing",
            "probabilities": {
                "billing": 1,
                "tech": 0,
                "other": 0
            },
            "confidence": 0.9997,
            "answer_confidence": 1,
            "action": {
                "act_probability": 1
            }
        },
        "urgent": {
            "type": "noul",
            "noul": 0.8202,
            "confidence": 0.8202,
            "answer_confidence": 0.8202,
            "action": {
                "act_probability": 1
            }
        },
        "anger": {
            "type": "score",
            "score": 1.358,
            "legend": {
                "0": "不急",
                "1": "尽快",
                "2": "立刻"
            },
            "probabilities": {
                "0": 0.0299,
                "1": 0.5822,
                "2": 0.3879
            },
            "confidence": 0.2835,
            "answer_confidence": 0.5822,
            "action": {
                "act_probability": 1
            }
        }
    },
    "usage": {
        "input_tokens": 162,
        "output_tokens": 0
    },
    "routing": {
        "model": "multilingual",
        "repo": "convaiinnovations/laya/multilingual",
        "reason": "non-Latin script (han, 100% of letters); the English checkpoint cannot read it",
        "detection": {
            "script": "han",
            "script_profile": {
                "han": 1
            },
            "language": null,
            "is_english": false,
            "language_undecided": true,
            "diacritic_rate": 0,
            "non_latin_fraction": 1
        },
        "workflow": null
    }
}

Kev

复制代码

curl -sS http://127.0.0.1:18008/v1/systemone \
  -H 'content-type: application/json' \
  -d '{
    "state": "订单晚了两周，鞋码也不对，卡里还被扣了两次。",
    "model": "kev-latest",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "哪个团队来处理？",
        "criteria": {
          "returns": "退换货、尺码或损坏",
          "shipping": "物流延误、丢件",
          "billing": "扣款、发票、支付"
        }
      },
      "escalate": {"type": "noul", "instructions": "需要马上人工处理吗？"},
      "frustration": {
        "type": "score",
        "instructions": "客户挫败感有多强？",
        "criteria": ["平静", "不满", "非常生气"]
      }
    }
  }'

字段和 Laya 一样。latency_ms 是这一次花了多少毫秒。CPU 上会比你的 Mac 慢。

bash 复制代码

响应结果：
{
    "model": "kev-latest",
    "answers": {
        "department": {
            "type": "choice",
            "choice": "returns",
            "confidence": 0.1315,
            "probabilities": {
                "returns": 0.421,
                "shipping": 0.4068,
                "billing": 0.1722
            }
        },
        "escalate": {
            "type": "noul",
            "noul": 0.6857
        },
        "frustration": {
            "type": "score",
            "score": 1.4159,
            "legend": {
                "0": "平静",
                "1": "不满",
                "2": "非常生气"
            },
            "probabilities": {
                "0": 0.0479,
                "1": 0.4884,
                "2": 0.4637
            },
            "confidence": 0.7442
        }
    },
    "usage": {
        "input_tokens": 87,
        "output_tokens": 212
    },
    "latency_ms": 1358.8
}

NanoJev

是非题写 boolean，看 p_true。

复制代码

curl -sS http://127.0.0.1:18111/api/evaluate \
  -H 'content-type: application/json' \
  -d '{
    "states": [
      {
        "id": "工单1",
        "state": "三月被扣了两次款，请今天退款。",
        "questions": {
          "department": {
            "type": "choice",
            "instructions": "这个工单应该交给哪个部门？",
            "criteria": {
              "billing": "发票、扣款、退款",
              "tech": "故障和 bug",
              "other": "其他事项"
            }
          },
          "billing": {
            "type": "boolean",
            "instructions": "这是不是账单问题？",
            "criteria": {"true": "明确在说扣款或退款", "false": "与扣款无关"}
          },
          "anger": {
            "type": "score",
            "instructions": "客户有多着急？",
            "criteria": ["不急", "尽快", "立刻"]
          }
        }
      }
    ]
  }'

结论在 states[0].answers。

bash 复制代码

响应结果：

{
    "schema_version": "openjev-toy-inference-v1",
    "checkpoint": {
        "directory": "/opt/decision-models/models/NanoJev",
        "base_model": "Qwen/Qwen3-0.6B",
        "base_revision": "c1899de289a04d12100db370d81485cdf75e47ca",
        "set_head": "attention"
    },
    "temperature": {
        "value": 1,
        "fitted_by_this_command": false,
        "note": "显式应用给定标量；默认1不表示模型已校准。"
    },
    "execution": {
        "device": "cpu",
        "parameter_storage": "float32",
        "precision": "fp32",
        "forward_autocast": "disabled",
        "states": 1,
        "questions": 3,
        "candidate_paths": 7,
        "forward_passes": 1,
        "batch_questions_limit": "all",
        "autoregressive_decode_steps": 0,
        "prefix_sharing": false,
        "max_length": 8192,
        "disable_native_triton": true,
        "network_model_calls": 0,
        "persistent_model_load_count": 1,
        "inference_call_index": 5,
        "note": "上游入口只接受 CUDA；本进程在 Apple MPS 上用同一份权重做本地推理"
    },
    "states": [
        {
            "id": "工单1",
            "answers": {
                "department": {
                    "type": "choice",
                    "probabilities": {
                        "billing": 0.33446261286735535,
                        "tech": 0.4972914159297943,
                        "other": 0.16824595630168915
                    },
                    "choice": "tech",
                    "value": "tech"
                },
                "billing": {
                    "type": "boolean",
                    "probabilities": {
                        "false": 0.15340343117713928,
                        "true": 0.8465965986251831
                    },
                    "p_true": 0.8465965986251831,
                    "value": true
                },
                "anger": {
                    "type": "score",
                    "probabilities": {
                        "0": 0.18612158298492432,
                        "1": 0.41676998138427734,
                        "2": 0.3971083462238312
                    },
                    "score": 1.2109866738319397,
                    "level": 1,
                    "value": 1.2109866738319397
                }
            }
        }
    ]
}

服务器上这次的实测结果

Laya：部门 billing，概率 0.9997；"是否今天处理"是 0.65。

Kev： "需要马上人工处理吗"是 0.69，这一次约 1 秒。

NanoJev："是不是账单问题"的 p_true 是 0.85，value 为 true。

bash 复制代码

三个服务的安装位置：
服务	配置文件
Laya
/etc/systemd/system/laya.service
Kev
/etc/systemd/system/kev.service
NanoJev
/etc/systemd/system/nanojev.service

bash 复制代码

systemctl status laya kev nanojev

结论

我的乌班图服务器是8核心16G的配置，没有显卡，接口响应时间再1秒内差不多。