Files
pyannote-audio/tutorials/applying_a_pipeline.ipynb
T

36 KiB

Applying a pretrained pipeline

In this tutorial, you will learn how to apply pyannote.audio pipelines on an audio file.

A pipeline takes an audio file as input and returns a labeled temporal segmentation of the audio file.

More precisely, it usually applies a pretrained model (= neural network) on the audio file, post-processes the output of the model, and returns its output as a pyannote.core.Annotation instance. It should become clearer as you keep reading...

Loading pipeline from 🤗 hub

Pretrained pipelines are available on 🤗 Huggingface model hub and can be listed by looking for the pyannote-audio-pipeline tag.

In [ ]:
from huggingface_hub import HfApi
available_pipelines = [p.modelId for p in HfApi().list_models(filter="pyannote-audio-pipeline")]
available_pipelines
['pyannote/speaker-diarization',
 'pyannote/speaker-segmentation',
 'pyannote/voice-activity-detection',
 'pyannote/overlapped-speech-detection',
 'philschmid/pyannote-speaker-diarization-endpoint']

To load the speaker diarization pipeline,

In [ ]:
from huggingface_hub import notebook_login
notebook_login()
Login successful
Your token has been saved to /Users/hbredin/.huggingface/token
Authenticated through git-credential store but this isn't the helper defined on your machine.
You might have to re-authenticate when pushing to the Hugging Face Hub. Run the following command in your terminal in case you want to set this credential helper as the default

git config --global credential.helper store
In [ ]:
from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained("pyannote/speaker-diarization@develop", use_auth_token=True)

Processing a file from disk

... and apply it to an audio file.

The pipeline will automatically use GPUs when available. On CPU it might take a long while (up to 10x RT).

In [ ]:
ROOT_DIR = "/Users/hbredin/Development/pyannote/pyannote-audio"
AUDIO_FILE = f"{ROOT_DIR}/tutorials/assets/sample.wav"
dia = pipeline(AUDIO_FILE)

Visualizing the output

Most pipelines return a pyannote.core.Annotation instance...

In [ ]:
from pyannote.core import Annotation
assert isinstance(dia, Annotation)

... whose API you can use to print the result:

In [ ]:
for speech_turn, track, speaker in dia.itertracks(yield_label=True):
    print(f"{speech_turn.start:4.1f} {speech_turn.end:4.1f} {speaker}")
 6.7  7.1 SPEAKER_02
 6.9  8.3 SPEAKER_00
 8.3 14.8 SPEAKER_02
 9.8 11.0 SPEAKER_00
14.3 18.5 SPEAKER_01
17.9 21.5 SPEAKER_02
21.7 28.6 SPEAKER_01
27.8 29.5 SPEAKER_02

If you happen to be running this example in a Jupyter notebook, dia can be visualized directly:

In [ ]:
# we visualize [0, 30] time range
from pyannote.core import notebook, Segment
notebook.crop = Segment(0, 30)
dia

When available, the reference annotation can be visualized too, for comparison:

In [ ]:
from pyannote.database.util import load_rttm
REFERENCE = f"{ROOT_DIR}/tutorials/assets/sample.rttm"
reference = load_rttm(REFERENCE)["sample"]

# map hypothesized and reference speakers for visualization purposes
pipeline.optimal_mapping(dia, reference)

Processing a file from memory

In case the audio file is not stored on disk, pipelines can also process audio provided as a {"waveform": ..., "sample_rate": ...} dictionary.

In [ ]:
import torchaudio
waveform, sample_rate = torchaudio.load(AUDIO_FILE)

print(f"{type(waveform)=}")
print(f"{waveform.shape=}")
print(f"{waveform.dtype=}")

audio_in_memory = {"waveform": waveform, "sample_rate": sample_rate}
type(waveform)=<class 'torch.Tensor'>
waveform.shape=torch.Size([1, 480000])
waveform.dtype=torch.float32
In [ ]:
vad = Pipeline.from_pretrained("pyannote/voice-activity-detection")
vad(audio_in_memory)