Teams Bot机器人实时语音识别的多引擎的处理

之前开发的Teams的Bot 是使用微软的实时语音识别。现在增加了一个选项，可以在Azure 和 Soniox两引擎间切换。

Azure 的实时语音识别

Azure 的实时语音识别使用 Microsoft.CognitiveServices.Speech.Transcription 进行处理。这个质量很不错，而且可以设置每个人的语音特征（wav audio file for creating voice signatures must be 16-bit, 16 kHz sample rate, in single channel (mono) format. The recommended length for each audio sample is between 30 seconds and two minutes. ）。

cs 复制代码

            byte[] fileBytes = File.ReadAllBytes(fn);
            var content = new ByteArrayContent(fileBytes);
            var client = new HttpClient();
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);
            var response = await client.PostAsync($"https://signature.{region}.cts.speech.microsoft.com/api/v1/Signature/GenerateVoiceSignatureFromByteArray", content);

            var jsonData = await response.Content.ReadAsStringAsync();
            var result = JsonConvert.DeserializeObject<VoiceSignature>(jsonData);
            return JsonConvert.SerializeObject(result.Signature);

设置了语音特征后就可以在实时语音识别中，自动识别说话者。一切都好，唯一的一个问题就是不便宜，目前一个月的费用有几万美元。

Soniox的实时语音识别

Soniox的实时语音识别引擎，是用WebSocket的，这一点和Azure的差别很大。下面的代码就是初始化识别引擎连接。

cs 复制代码

 ClientWebSocket ws= new ClientWebSocket();
 ws.ConnectAsync(new Uri(ConversationMgr.instance.SonioxUrl), CancellationToken.None).Wait();

 // Send start request with correct field names
 var startMessage = Encoding.UTF8.GetBytes(Newtonsoft.Json.JsonConvert.SerializeObject(new
 {
     api_key = ConversationMgr.instance.SonioxKey,
     audio_format = ConversationMgr.instance.SonioxAudioFormat,
     sample_rate = int.Parse(ConversationMgr.instance.SonioxSampleRate),
     num_channels = int.Parse(ConversationMgr.instance.SonioxNumChannels),
     model = ConversationMgr.instance.SonioxModel,
     enable_speaker_diarization=true,
     language_hints = ConversationMgr.instance.SonioxLanguageHints.Replace(" ", "").Split(new string[] { "," }, StringSplitOptions.RemoveEmptyEntries)
 }));

 ws.SendAsync(new ArraySegment<byte>(startMessage), WebSocketMessageType.Text, true, CancellationToken.None).Wait();

初始化之后，当收到从机器人收到语音包后，就把包的时间发送到Soniox。

var audioChunk = new ArraySegment<byte>(item.Buff, 0, item.Buff.Length);

await ws.SendAsync(audioChunk, WebSocketMessageType.Binary, true, CancellationToken.None);

同时并行一个任务从 Soniox接受识别的结果：

result = await ws.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);

返回的结果如下

javascript 复制代码

{"tokens":[{"text":"Wh","start_ms":540,"end_ms":540,"confidence":0.999,"is_final":false,"speaker":"1"},{"text":"at","start_ms":540,"end_ms":600,"confidence":1,"is_final":false,"speaker":"1"},{"text":" is","start_ms":660,"end_ms":720,"confidence":0.998,"is_final":false,"speaker":"1"},{"text":" y","start_ms":780,"end_ms":840,"confidence":1,"is_final":false,"speaker":"1"},{"text":"our","start_ms":840,"end_ms":900,"confidence":1,"is_final":false,"speaker":"1"},{"text":" best","start_ms":960,"end_ms":1020,"confidence":1,"is_final":false,"speaker":"1"}],"final_audio_proc_ms":0,"total_audio_proc_ms":1800}

{

"tokens": [

{

"text": "Wh",

"start_ms": 540,

"end_ms": 540,

"confidence": 0.999,

"is_final": false,

"speaker": "1"

{

"text": "at",

"start_ms": 540,

"end_ms": 600,

"confidence": 1,

"is_final": false,

"speaker": "1"

{

"text": " is",

"start_ms": 660,

"end_ms": 720,

"confidence": 0.998,

"is_final": false,

"speaker": "1"

{

"text": " y",

"start_ms": 780,

"end_ms": 840,

"confidence": 1,

"is_final": false,

"speaker": "1"

{

"text": "our",

"start_ms": 840,

"end_ms": 900,

"confidence": 1,

"is_final": false,

"speaker": "1"

{

"text": " best",

"start_ms": 960,

"end_ms": 1020,

"confidence": 1,

"is_final": false,

"speaker": "1"

}

"final_audio_proc_ms": 0,

"total_audio_proc_ms": 1800

}