Add kNN-VC

[eginhard]

New voice conversion model

• Paper: https://www.isca-archive.org/interspeech_2023/baas23_interspeech.html
• Original code: https://github.com/bshall/knn-vc

Usage

from TTS.api import TTS

knnvc = TTS("voice_conversion_models/multilingual/multi-dataset/knnvc")
knnvc.voice_conversion_to_file("source.wav", ["target1.wav", "target2.wav", ...], "output.wav")

By default the prematched vocoder is used. For the non-prematched version:

knnvc = TTS(
    "voice_conversion_models/multilingual/multi-dataset/knnvc", 
    vocoder_name="vocoder_models/en/librispeech100/wavlm-hifigan"
)

CLI:

tts --model_name voice_conversion_models/multilingual/multi-dataset/knnvc \
    --source_wav source.wav \
    --target_wav target1.wav target2.wav ...

Internals

The vocoder models are the same as the original ones, but the model names and configs were modified with the following code to be compatible with Coqui and republished in https://github.com/idiap/coqui-ai-TTS/releases/tag/v0.25.2_models:

import torch
from TTS.vocoder.configs.hifigan_config import HifiganConfig

m = torch.load("prematch_g_02500000.pt", map_location="cpu")
m["model_disc"] = None
m["model"] = m.pop("generator")
torch.save(m, "model.pth")

hifigan_config = HifiganConfig()
hifigan_config["audio"]["sample_rate"] = 16000
hifigan_config["generator_model_params"]["inference_padding"] = 0
hifigan_config["generator_model_params"]["pre_linear"] = 1024
hifigan_config["generator_model_params"]["in_channels"] = 512
hifigan_config["generator_model_params"]["upsample_factors"] = [10, 8, 2, 2]
hifigan_config["generator_model_params"]["upsample_kernel_sizes"] = [20, 16, 4, 4]
hifigan_config.save_json("config.json")