Files
pyannote-audio/tutorials/community/offline_usage_speaker_diarization.ipynb
T

7.4 KiB

Offline Speaker Diarization (speaker-diarization-3.1)

This notebooks gives a short introduction how to use the speaker-diarization-3.1 pipeline with local models.

In order to use local models, you first need to download them from huggingface and place them in a local folder. Then you need to create a local config file, similar to the one in HF, but with local model paths.

❗ Naming of the model files is REALLY important! See end of notebook for details. ❗

Get the models

  1. Install the pyannote-audio package: !pip install pyannote.audio
  2. Create a huggingface account https://huggingface.co/join
  3. Accept pyannote/segmentation-3.0 user conditions
  4. Create a local folder models, place all downloaded files there
    1. wespeaker-voxceleb-resnet34-LM, to be placed in models/pyannote_model_wespeaker-voxceleb-resnet34-LM.bin
    2. segmentation-3.0, to be placed in models/pyannote_model_segmentation-3.0.bin

Running ls models should show the following files:

pyannote_model_segmentation-3.0.bin (5.7M)
pyannote_model_wespeaker-voxceleb-resnet34-LM.bin (26MB)

❗ make sure the 'wespeaker-voxceleb-resnet34-LM' model is named 'pyannote_model_wespeaker-voxceleb-resnet34-LM.bin' ❗

Config for local models

Create a local config, similar to the one in HF: speaker-diarization-3.1/blob/main/config.yaml, but with local model paths

Contents of models/pyannote_diarization_config.yaml:

version: 3.1.0

pipeline:
  name: pyannote.audio.pipelines.SpeakerDiarization
  params:
    clustering: AgglomerativeClustering
    # embedding: pyannote/wespeaker-voxceleb-resnet34-LM  # if you want to use the HF model
    embedding: models/pyannote_model_wespeaker-voxceleb-resnet34-LM.bin  # if you want to use the local model
    embedding_batch_size: 32
    embedding_exclude_overlap: true
    # segmentation: pyannote/segmentation-3.0  # if you want to use the HF model
    segmentation: models/pyannote_model_segmentation-3.0.bin  # if you want to use the local model
    segmentation_batch_size: 32

params:
  clustering:
    method: centroid
    min_cluster_size: 12
    threshold: 0.7045654963945799
  segmentation:
    min_duration_off: 0.0

Loading the local pipeline

Hint: The paths in the config are relative to the current working directory, not relative to the config file. If you want to start your notebook/script from a different directory, you can use os.chdir temporarily, to 'emulate' config-relative paths.

In [ ]:
from pathlib import Path
from pyannote.audio import Pipeline

def load_pipeline_from_pretrained(path_to_config: str | Path) -> Pipeline:
    path_to_config = Path(path_to_config)

    print(f"Loading pyannote pipeline from {path_to_config}...")
    # the paths in the config are relative to the current working directory
    # so we need to change the working directory to the model path
    # and then change it back

    cwd = Path.cwd().resolve()  # store current working directory

    # first .parent is the folder of the config, second .parent is the folder containing the 'models' folder
    cd_to = path_to_config.parent.parent.resolve()

    print(f"Changing working directory to {cd_to}")
    os.chdir(cd_to)

    pipeline = Pipeline.from_pretrained(path_to_config)

    print(f"Changing working directory back to {cwd}")
    os.chdir(cwd)

    return pipeline

PATH_TO_CONFIG = "path/to/your/pyannote_diarization_config.yaml"
pipeline = load_pipeline_from_pretrained(PATH_TO_CONFIG)

Notes on file naming (pyannote-audio 3.1.1)

Pyannote uses some internal logic to determine the model type.

The funtion def PretrainedSpeakerEmbedding(... in (speaker_verification.py)[https://github.com/pyannote/pyannote-audio/blob/develop/pyannote/audio/pipelines/speaker_verification.py#L712] uses the the file path of the model to infer the model type.

def PretrainedSpeakerEmbedding(
    embedding: PipelineModel,
    device: torch.device = None,
    use_auth_token: Union[Text, None] = None,
):
    #...
    if isinstance(embedding, str) and "pyannote" in embedding:
        return PyannoteAudioPretrainedSpeakerEmbedding(
            embedding, device=device, use_auth_token=use_auth_token
        )

    elif isinstance(embedding, str) and "speechbrain" in embedding:
        return SpeechBrainPretrainedSpeakerEmbedding(
            embedding, device=device, use_auth_token=use_auth_token
        )

    elif isinstance(embedding, str) and "nvidia" in embedding:
        return NeMoPretrainedSpeakerEmbedding(embedding, device=device)

    elif isinstance(embedding, str) and "wespeaker" in embedding:
        return ONNXWeSpeakerPretrainedSpeakerEmbedding(embedding, device=device)  # <-- this is called, but the wespeaker-voxceleb-resnet34-LM is not an ONNX model

    else:
        # fallback to pyannote in case we are loading a local model
        return PyannoteAudioPretrainedSpeakerEmbedding(
            embedding, device=device, use_auth_token=use_auth_token
        )

The wespeaker-voxceleb-resnet34-LM model is not an ONNX model, but a PyannoteAudioPretrainedSpeakerEmbedding. So if wespeaker is in the file name, the code will infer the model type incorrectly. If pyannote is somewhere in the file name, the model type will be inferred correctly, as the first if statement will be true...