2023-09-20 17:39:30 +02:00
2024-11-29 10:19:17 +01:00
2023-03-16 15:04:26 +01:00
2020-10-12 16:20:50 +02:00

Using pyannote.audio open-source toolkit in production? Consider switching to pyannoteAI for better and faster options.

pyannote speaker diarization toolkit

pyannote.audio is an open-source toolkit written in Python for speaker diarization. Based on PyTorch machine learning framework, it comes with state-of-the-art pretrained models and pipelines, that can be further finetuned to your own data for even better performance.

Highlights

pyannote.audio open-source speaker diarization pipeline

  1. Install pyannote.audio with pip install pyannote.audio
  2. Accept pyannote/speaker-diarization-4.0 user conditions
  3. Create Huggingface access token at hf.co/settings/tokens.
import torch
from pyannote.audio import Pipeline
from pyannote.audio.pipelines.utils.hook import ProgressHook

# Open-source pyannote speaker diarization pipeline
pipeline = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-4.0",
    token="HUGGINGFACE_ACCESS_TOKEN")

# send pipeline to GPU (when available)
pipeline.to(torch.device("cuda"))

# apply pretrained pipeline (with optional progress hook)
with ProgressHook() as hook:
    diarization = pipeline("audio.wav", hook=hook)  # runs locally

# print the result
for turn, _, speaker in diarization.itertracks(yield_label=True):
    print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker_{speaker}")
# start=0.2s stop=1.5s speaker_0
# start=1.8s stop=3.9s speaker_1
# start=4.2s stop=5.7s speaker_0
# ...

pyannoteAI premium speaker diarization pipeline

  1. Create pyannoteAI API key at dashboard.pyannote.ai
  2. Enjoy 150 hours of free credits!
from pyannote.audio import Pipeline

# Premium pyannoteAI speaker diarization service
pipeline = Pipeline.from_pretrained(
    "pyannoteAI/speaker-diarization-precision", token="PYANNOTEAI_API_KEY")

diarization = pipeline("audio.wav")  # runs on pyannoteAI servers

# print the result
for turn, _, speaker in diarization.itertracks(yield_label=True):
    print(f"start={turn.start:.1f}s stop={turn.end:.1f}s {speaker}")
# start=0.2s stop=1.6s SPEAKER_00
# start=1.8s stop=4.0s SPEAKER_01 
# start=4.2s stop=5.6s SPEAKER_00
# ...

Visit docs.pyannote.ai to learn about other pyannoteAI features (voiceprinting, confidence scores, ...)

Benchmark

Out of the box, pyannote.audio speaker diarization pipeline v4.0 is expected to be much better than v3.1.

pyannoteAI premium models are even better (and also 2x faster). labs model is currently in private beta.

Benchmark (last updated in 2025-08)
v3.1

v4.0

API

labs
AISHELL-4 12.2 11.7 11.8 11.4
AliMeeting (channel 1) 24.5 20.3 16.3 15.2
AMI (IHM) 18.8 17.0 13.2 12.9
AMI (SDM) 22.7 19.9 15.8 15.6
AVA-AVD 49.7 44.6 40.7 37.1
CALLHOME (part 2) 28.5 26.7 17.6 16.6
DIHARD 3 (full) 21.4 20.2 15.7 14.7
Ego4D (dev.) 51.2 46.8 44.7 39.0
MSDWild 25.4 22.8 17.9 17.3
RAMC 22.2 20.8 10.6 10.5
REPERE (phase2) 7.9 8.9 7.3 7.4
VoxConverse (v0.3) 11.2 11.2 9.0 8.5

Diarization error rate (in %, the lower, the better)

Benchmark (last updated in 2025-08) Speed up
AMI (IHM), ~1h files 31s per hour of audio 14s per hour of audio 2.2x faster
DIHARD 3 (full), ~5min files 37s per hour of audio 14s per hour of audio 2.6x faster

Processing speed on a NVIDIA H100 80GB HBM3

Documentation

Citations

If you use pyannote.audio please use the following citations:

@inproceedings{Plaquet23,
  author={Alexis Plaquet and Hervé Bredin},
  title={{Powerset multi-class cross entropy loss for neural speaker diarization}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}
@inproceedings{Bredin23,
  author={Hervé Bredin},
  title={{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}

Development

The commands below will setup pre-commit hooks and packages needed for developing the pyannote.audio library.

pip install -e .[dev,testing]
pre-commit install

Test

pytest
Languages
Jupyter Notebook 63.6%
Python 36.4%