深度学习系列68：声音克隆项目OpenVoice

IE062024-05-15 18:04

1. 项目介绍

OpenVoice 是 myshell ai 开源的一款基于人工智能技术的语音克隆工具。其核心功能是通过提供发言者的短音频片段（参考语音），实现声音的高效克隆。这意味着您可以使用OpenVoice来克隆任何人的声音，而且不限于特定语言。无论您是想要模仿某位名人的声音，还是需要在不同语言之间进行语音转换，OpenVoice都能够满足您的需求。

OpenVoice还可以实现音色克隆和控制。

项目安装还算简单，首先安装项目文件夹：git clone https://github.com/myshell-ai/OpenVoice.git

然后安装依赖包：pip install -r requirements.txt

然后下载模型：https://link.zhihu.com/?target=https%3A//myshell-public-repo-hosting.s3.amazonaws.com/checkpoints_1226.zip。如果连不上，也可以去hf镜像站下载：https://hf-mirror.com/myshell-ai/OpenVoice

2. 语音拷贝代码

复制代码

import os
import torch
from openvoice import se_extractor
from openvoice.api import BaseSpeakerTTS, ToneColorConverter
from pydub import AudioSegment,playback
ckpt_base = 'checkpoints/base_speakers/EN'
ckpt_converter = 'checkpoints/converter'
device="cuda:0" if torch.cuda.is_available() else "cpu"
output_dir = 'outputs'

base_speaker_tts = BaseSpeakerTTS(f'{ckpt_base}/config.json', device=device)
base_speaker_tts.load_ckpt(f'{ckpt_base}/checkpoint.pth')

tone_color_converter = ToneColorConverter(f'{ckpt_converter}/config.json', device=device)
tone_color_converter.load_ckpt(f'{ckpt_converter}/checkpoint.pth')

os.makedirs(output_dir, exist_ok=True)
source_se = torch.load(f'{ckpt_base}/en_default_se.pth').to(device)
reference_speaker = 'resources/example_reference.mp3' # This is the voice you want to clone
target_se, audio_name = se_extractor.get_se(reference_speaker, tone_color_converter, target_dir='processed', vad=True)
save_path = f'{output_dir}/output_en_default.wav'

# Run the base speaker tts。这里也可以换成你自己的tts
text = "hello china. Today is very good."
src_path = f'{output_dir}/tmp.wav'
base_speaker_tts.tts(text, src_path, speaker='default', language='English', speed=1.0)

# Run the tone color converter
encode_message = "@MyShell"
tone_color_converter.convert(
    audio_src_path=src_path, 
    src_se=source_se, 
    tgt_se=target_se, 
    output_path=save_path,
    message=encode_message)
playback.play(AudioSegment.from_wav(save_path))