Files
2026-03-03 14:52:19 +09:00

18 KiB
Executable File

Demo

Using pre-trained model (facebook/wav2vec2-large-960h-lv60-self)

In [1]:
audio_path = "dataset/hensen/umbrella/umbrella.wav"
lyrics_path = "dataset/hensen/umbrella/umbrella.txt"
In [2]:
from lsync import LyricsSync

lsync = LyricsSync()
words, lrc = lsync.sync(audio_path, lyrics_path)
print(lrc)
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-large-960h-lv60-self and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
[00:00.00] <00:27.53> you <00:27.90> have <00:28.21> my <00:28.42> heart
[00:29.40] <00:29.46> and <00:29.62> we'll <00:29.98> never <00:30.32> be <00:30.62> worlds <00:31.12> apart
[00:32.32] <00:32.35> maybe <00:33.07> in <00:33.38> magazines
[00:35.11] <00:35.16> but <00:35.48> you'll <00:35.75> still <00:36.14> be <00:36.39> my <00:36.61> star
[00:37.84] <00:37.88> baby <00:38.38> 'cause <00:38.98> in <00:39.20> the <00:39.42> dark
[00:40.57] <00:40.61> you <00:40.92> can't <00:41.30> see <00:41.61> shiny <00:42.14> cars
[00:43.14] <00:43.17> and <00:43.35> that's <00:43.70> when <00:44.03> you <00:44.38> need <00:44.74> me <00:44.96> there
[00:46.06] <00:46.10> with <00:46.46> you <00:46.88> i'll <00:47.16> always <00:47.82> share
[00:49.14] <00:49.17> because <00:50.56> when <00:50.74> the <00:50.92> sun <00:51.21> shines <00:51.64> we <00:51.93> shine <00:52.32> together
[00:53.14] <00:53.38> told <00:53.74> you <00:54.10> i'll <00:54.42> be <00:54.78> here <00:55.06> forever
[00:56.03] <00:56.10> said <00:56.56> i'll <00:56.92> always <00:57.50> be <00:57.85> your <00:57.98> friend
[00:58.50] <00:58.53> took <00:58.75> an <00:58.96> oath <00:59.20> gonna <00:59.46> stick <00:59.78> it <01:00.02> out <01:00.21> till <01:00.57> the <01:00.84> end
[01:01.56] <01:01.62> now <01:01.92> that <01:02.10> it's <01:02.34> rainin' <01:03.00> more <01:03.30> than <01:03.57> ever
[01:04.32] <01:04.34> know <01:04.71> that <01:05.07> we'll <01:05.37> still <01:05.78> have <01:06.14> each <01:06.34> other
[01:07.09] <01:07.14> you <01:07.29> can <01:07.46> stand <01:07.85> under <01:08.18> my <01:08.62> umbrella
[01:09.84] <01:09.87> you <01:10.01> can <01:10.18> stand <01:10.60> under <01:10.92> my <01:11.35> umbrella
[01:12.62] <01:12.71> ella <01:13.37> ella <01:14.15> ae <01:14.76> ae <01:14.98> ae
[01:16.00] <01:16.06> under <01:16.43> my <01:16.90> umbrella <01:18.20> ella <01:18.87> ella <01:19.64> ae <01:20.32> ae <01:21.00> ae
[01:21.54] <01:21.60> under <01:21.93> my <01:22.40> umbrella <01:23.70> ella <01:24.37> ella <01:25.15> ae <01:25.79> ae <01:26.51> ae
[01:27.04] <01:27.10> under <01:27.48> my <01:27.92> umbrella <01:29.21> ella <01:29.92> ella <01:30.65> ae <01:31.32> ae <01:32.01> ae <01:32.07> ae <01:32.68> ae <01:33.35> ae
[01:33.56] <01:33.60> these <01:34.00> fancy <01:34.50> things
[01:35.65] <01:35.70> will <01:36.06> never <01:36.37> come <01:36.84> in <01:37.06> between
[01:38.37] <01:38.42> you're <01:38.76> part <01:39.01> of <01:39.18> my <01:39.57> entity
[01:41.14] <01:41.18> here <01:41.54> for <01:42.00> infinity
[01:43.90] <01:43.93> when <01:44.12> the <01:44.32> war <01:44.68> has <01:44.98> took <01:45.32> it's <01:45.56> part
[01:46.65] <01:46.68> when <01:46.85> the <01:47.06> world <01:47.37> has <01:47.74> dealt <01:48.07> it's <01:48.34> cards
[01:49.78] <01:49.90> if <01:50.15> the <01:50.48> hand <01:50.87> is <01:51.01> hard
[01:52.35] <01:52.37> together <01:52.90> we'll <01:53.21> mend <01:53.57> your <01:53.79> heart
[01:55.24] <01:55.28> because <01:56.68> when <01:56.85> the <01:57.04> sun <01:57.35> shines <01:57.76> we <01:58.06> shine <01:58.43> together
[01:59.24] <01:59.40> told <01:59.87> you <02:00.26> i'll <02:00.51> be <02:00.84> here <02:01.15> forever
[02:02.15] <02:02.20> said <02:02.68> i'll <02:03.01> always <02:03.62> be <02:03.96> your <02:04.07> friend
[02:04.64] <02:04.65> took <02:04.84> an <02:05.07> oath <02:05.29> gonna <02:05.56> stick <02:05.90> it <02:06.12> out <02:06.34> till <02:06.70> the <02:06.93> end
[02:07.68] <02:07.70> now <02:08.06> that <02:08.19> it's <02:08.46> rainin' <02:09.12> more <02:09.42> than <02:09.69> ever
[02:10.46] <02:10.47> know <02:10.84> that <02:11.19> we'll <02:11.52> still <02:11.87> have <02:12.25> each <02:12.46> other
[02:13.21] <02:13.25> you <02:13.42> can <02:13.59> stand <02:13.97> under <02:14.30> my <02:14.84> umbrella
[02:15.93] <02:15.97> you <02:16.14> can <02:16.28> stand <02:16.70> under <02:17.03> my <02:17.50> umbrella
[02:18.74] <02:18.80> ella <02:19.47> ella <02:20.25> ae <02:20.92> ae <02:21.62> ae
[02:22.12] <02:22.18> under <02:22.56> my <02:23.02> umbrella <02:24.30> ella <02:25.00> ella <02:25.75> ae <02:26.46> ae <02:27.15> ae
[02:27.68] <02:27.74> under <02:28.08> my <02:28.52> umbrella <02:29.86> ella <02:30.47> ella <02:31.18> ae <02:31.90> ae <02:32.62> ae
[02:33.14] <02:33.20> under <02:33.58> my <02:34.02> umbrella <02:35.34> ella <02:36.00> ella <02:36.78> ae <02:37.43> ae <02:38.15> ae
[02:38.65] <02:38.78> ae <02:39.47> ae <02:39.86> ae
[02:40.12] <02:40.15> you <02:40.42> can <02:40.70> run <02:41.12> into <02:41.71> my <02:42.25> arms
[02:42.86] <02:42.96> it's <02:43.25> okay <02:43.78> don't <02:44.12> be <02:44.53> alarmed <02:45.31> come <02:45.78> into <02:46.70> me
[02:47.14] <02:47.15> there's <02:47.50> no <02:47.75> distance <02:48.59> in <02:49.12> between <02:50.59> our <02:50.93> love
[02:51.47] <02:51.50> so <02:51.81> go <02:52.08> on <02:52.24> and <02:52.50> let <02:52.84> the <02:53.20> rain <02:53.87> pour
[02:54.64] <02:54.71> i'll <02:54.96> be <02:55.42> all <02:55.65> you <02:55.97> need <02:56.37> and <02:56.70> more
[02:59.74] <03:01.43> because <03:02.80> when <03:02.96> the <03:03.15> sun <03:03.46> shines <03:03.87> we <03:04.18> shine <03:04.53> together
[03:05.56] <03:05.59> told <03:05.97> you <03:06.37> i'll <03:06.64> be <03:07.03> here <03:07.28> forever
[03:08.30] <03:08.34> said <03:08.78> i'll <03:09.18> always <03:09.75> be <03:10.21> your <03:10.34> friend <03:10.75> took <03:10.97> an <03:11.20> oath <03:11.43> gonna <03:11.78> stick <03:12.03> it <03:12.25> out <03:12.48> till <03:12.84> the <03:13.09> end
[03:13.81] <03:13.86> now <03:14.18> that <03:14.37> it's <03:14.75> rainin' <03:15.24> more <03:15.52> than <03:15.80> ever
[03:16.58] <03:16.62> know <03:16.96> that <03:17.31> we'll <03:17.62> still <03:18.00> have <03:18.36> each <03:18.58> other
[03:19.34] <03:19.36> you <03:19.53> can <03:19.70> stand <03:20.09> under <03:20.42> my <03:20.84> umbrella
[03:22.08] <03:22.09> you <03:22.25> can <03:22.42> stand <03:22.84> under <03:23.15> my <03:23.58> umbrella
[03:24.84] <03:24.92> ella <03:25.59> ella <03:26.37> ae <03:27.06> ae <03:27.74> ae
[03:28.25] <03:28.31> under <03:28.68> my <03:29.12> umbrella <03:30.42> ella <03:31.12> ella <03:31.86> ae <03:32.53> ae <03:33.24> ae
[03:33.75] <03:33.84> under <03:34.18> my <03:34.64> umbrella <03:35.92> ella <03:36.62> ella <03:37.37> ae <03:38.02> ae <03:38.75> ae
[03:39.25] <03:39.31> under <03:39.70> my <03:40.14> umbrella <03:41.46> ella <03:42.12> ella <03:42.59> ae <03:42.90> ae <03:43.56> ae <03:44.25> ae <03:44.90> ae <03:45.59> ae
[03:45.81] <03:45.98> it's <03:46.24> rainin' <03:46.90> raining
[03:47.86] <03:48.00> oh <03:48.28> baby <03:48.70> it's <03:48.96> rainin' <03:49.64> raining
[03:50.65] <03:50.70> baby <03:51.40> come <03:51.78> here <03:52.50> to <03:52.75> me
[03:53.25] <03:53.56> come <03:53.70> here <03:54.12> to <03:54.40> me
[03:54.48] <03:54.64> it's <03:55.58> raining <03:56.09> raining
[03:56.40] <03:56.46> oh <03:56.86> baby <03:56.96> it's <03:57.24> raining <03:57.65> raining
[03:57.80] <03:57.84> you <03:57.92> can <03:58.36> always <03:58.93> come <03:59.34> here

Using Fine-tuned model

Please update your model's id and path in lsync/config.py

In [3]:
YOUR_MODEL_ID = "en-finetuned-base"
YOUR_MODEL_BLANK_ID = 29
In [4]:
from lsync import LyricsSync

lsync = LyricsSync(lang=YOUR_MODEL_ID, blank_id=YOUR_MODEL_BLANK_ID)
words, lrc = lsync.sync(audio_path, lyrics_path)
print(lrc)
[00:00.00] <00:27.53> you <00:27.85> have <00:28.17> my <00:28.35> heart
[00:29.17] <00:29.46> and <00:29.60> we'll <00:29.96> never <00:30.28> be <00:30.58> worlds <00:30.98> apart
[00:31.87] <00:32.35> maybe <00:33.03> in <00:33.35> magazines
[00:35.07] <00:35.11> but <00:35.46> you'll <00:35.74> still <00:36.11> be <00:36.42> my <00:36.56> star
[00:37.38] <00:37.88> baby <00:38.50> 'cause <00:38.93> in <00:39.20> the <00:39.39> dark
[00:40.14] <00:40.61> you <00:40.92> can't <00:41.28> see <00:41.57> shiny <00:42.11> cars
[00:42.96] <00:43.20> and <00:43.35> that's <00:43.67> when <00:44.02> you <00:44.34> need <00:44.70> me <00:44.88> there
[00:45.60] <00:46.10> with <00:46.39> you <00:46.80> i'll <00:47.11> always <00:47.78> share
[00:48.56] <00:49.17> because <00:50.56> when <00:50.71> the <00:50.92> sun <00:51.21> shines <00:51.62> we <00:51.93> shine <00:52.30> together
[00:53.02] <00:53.34> told <00:53.70> you <00:54.06> i'll <00:54.39> be <00:54.71> here <00:55.03> forever
[00:55.80] <00:56.10> said <00:56.52> i'll <00:56.84> always <00:57.48> be <00:57.80> your <00:57.98> friend
[00:58.50] <00:58.52> took <00:58.74> an <00:58.92> oath <00:59.16> gonna <00:59.43> stick <00:59.74> it <00:59.89> out <01:00.20> till <01:00.53> the <01:00.75> end
[01:01.28] <01:01.60> now <01:01.92> that <01:02.10> it's <01:02.32> rainin' <01:02.96> more <01:03.30> than <01:03.52> ever
[01:04.06] <01:04.37> know <01:04.70> that <01:05.04> we'll <01:05.35> still <01:05.71> have <01:06.07> each <01:06.28> other
[01:06.82] <01:07.14> you <01:07.29> can <01:07.46> stand <01:07.84> under <01:08.15> my <01:08.56> umbrella
[01:09.54] <01:09.87> you <01:10.01> can <01:10.20> stand <01:10.57> under <01:10.90> my <01:11.29> umbrella
[01:12.42] <01:12.65> ella <01:13.35> ella <01:14.04> ae <01:14.70> ae <01:15.40> ae
[01:15.82] <01:16.04> under <01:16.40> my <01:16.78> umbrella <01:18.15> ella <01:18.85> ella <01:19.46> ae <01:20.20> ae <01:20.92> ae
[01:21.32] <01:21.57> under <01:21.92> my <01:22.32> umbrella <01:23.68> ella <01:24.34> ella <01:24.98> ae <01:25.70> ae <01:26.32> ae
[01:26.79] <01:27.07> under <01:27.43> my <01:27.82> umbrella <01:28.28> ella <01:29.18> ella <01:29.90> ae <01:30.06> ae <01:30.56> ae <01:31.21> ae <01:32.57> ae <01:33.29> ae
[01:33.54] <01:33.57> these <01:33.98> fancy <01:34.48> things
[01:35.21] <01:35.71> will <01:36.04> never <01:36.35> come <01:36.76> in <01:37.04> between
[01:37.92] <01:38.42> you're <01:38.76> part <01:39.00> of <01:39.15> my <01:39.50> entity
[01:40.71] <01:41.18> here <01:41.51> for <01:42.00> infinity
[01:43.37] <01:43.93> when <01:44.12> the <01:44.29> war <01:44.64> has <01:44.98> took <01:45.28> it's <01:45.51> part
[01:46.40] <01:46.70> when <01:46.85> the <01:47.01> world <01:47.35> has <01:47.71> dealt <01:48.06> it's <01:48.28> cards
[01:49.35] <01:49.84> if <01:50.14> the <01:50.43> hand <01:50.84> is <01:50.98> hard
[01:51.84] <01:52.37> together <01:52.90> we'll <01:53.21> mend <01:53.56> your <01:53.79> heart
[01:54.96] <01:55.28> because <01:56.68> when <01:56.84> the <01:57.04> sun <01:57.34> shines <01:57.74> we <01:58.04> shine <01:58.42> together
[01:59.46] <01:59.48> told <01:59.84> you <02:00.15> i'll <02:00.50> be <02:00.82> here <02:01.14> forever
[02:01.90] <02:02.21> said <02:02.62> i'll <02:02.93> always <02:03.57> be <02:03.92> your <02:04.07> friend
[02:04.60] <02:04.64> took <02:04.84> an <02:05.01> oath <02:05.28> gonna <02:05.62> stick <02:05.85> it <02:06.04> out <02:06.34> till <02:06.68> the <02:06.88> end
[02:07.43] <02:07.71> now <02:08.06> that <02:08.24> it's <02:08.43> rainin' <02:09.08> more <02:09.42> than <02:09.64> ever
[02:10.18] <02:10.50> know <02:10.81> that <02:11.18> we'll <02:11.50> still <02:11.84> have <02:12.21> each <02:12.40> other
[02:12.96] <02:13.25> you <02:13.42> can <02:13.58> stand <02:13.97> under <02:14.28> my <02:14.65> umbrella
[02:15.68] <02:16.00> you <02:16.12> can <02:16.24> stand <02:16.68> under <02:17.02> my <02:17.40> umbrella
[02:18.53] <02:18.78> ella <02:19.46> ella <02:20.14> ae <02:20.81> ae <02:21.43> ae
[02:21.93] <02:22.15> under <02:22.53> my <02:22.92> umbrella <02:24.28> ella <02:24.97> ella <02:25.65> ae <02:26.34> ae <02:27.06> ae
[02:27.43] <02:27.71> under <02:28.03> my <02:28.43> umbrella <02:29.84> ella <02:30.46> ella <02:31.15> ae <02:31.80> ae <02:32.52> ae
[02:32.90] <02:33.18> under <02:33.53> my <02:33.92> umbrella <02:35.30> ella <02:35.96> ella <02:36.68> ae <02:37.34> ae <02:38.06> ae
[02:38.62] <02:38.68> ae <02:39.42> ae <02:39.75> ae
[02:40.09] <02:40.15> you <02:40.42> can <02:40.68> run <02:41.08> into <02:41.68> my <02:42.08> arms
[02:42.84] <02:42.92> it's <02:43.21> okay <02:43.75> don't <02:44.08> be <02:44.43> alarmed <02:45.30> come <02:45.70> into <02:46.65> me
[02:47.40] <02:47.42> there's <02:47.64> no <02:47.70> distance <02:48.50> in <02:49.08> between <02:50.56> our <02:50.90> love
[02:51.43] <02:51.47> so <02:51.81> go <02:52.08> on <02:52.20> and <02:52.47> let <02:52.81> the <02:53.18> rain <02:53.81> pour
[02:54.53] <02:54.64> i'll <02:54.96> be <02:55.31> all <02:55.64> you <02:55.93> need <02:56.34> and <02:56.64> more
[03:01.37] <03:01.42> because <03:02.80> when <03:02.96> the <03:03.14> sun <03:03.46> shines <03:03.86> we <03:04.15> shine <03:04.52> together
[03:05.56] <03:05.58> told <03:05.93> you <03:06.28> i'll <03:06.62> be <03:06.96> here <03:07.28> forever
[03:08.03] <03:08.34> said <03:08.74> i'll <03:09.08> always <03:09.70> be <03:10.03> your <03:10.20> friend <03:10.75> took <03:10.96> an <03:11.15> oath <03:11.40> gonna <03:11.68> stick <03:11.98> it <03:12.18> out <03:12.46> till <03:12.80> the <03:13.02> end
[03:13.53] <03:13.84> now <03:14.18> that <03:14.36> it's <03:14.53> rainin' <03:15.18> more <03:15.52> than <03:15.75> ever
[03:16.30] <03:16.62> know <03:16.93> that <03:17.28> we'll <03:17.59> still <03:17.96> have <03:18.31> each <03:18.50> other
[03:19.06] <03:19.37> you <03:19.52> can <03:19.70> stand <03:20.08> under <03:20.37> my <03:20.78> umbrella
[03:21.78] <03:22.09> you <03:22.25> can <03:22.42> stand <03:22.80> under <03:23.14> my <03:23.52> umbrella
[03:24.65] <03:24.90> ella <03:25.59> ella <03:26.20> ae <03:26.92> ae <03:27.62> ae
[03:28.21] <03:28.28> under <03:28.65> my <03:29.02> umbrella <03:30.40> ella <03:31.08> ella <03:31.75> ae <03:32.42> ae <03:33.14> ae
[03:33.53] <03:33.81> under <03:34.14> my <03:34.53> umbrella <03:35.90> ella <03:36.58> ella <03:37.28> ae <03:37.92> ae <03:38.64> ae
[03:39.02] <03:39.30> under <03:39.65> my <03:40.06> umbrella <03:41.42> ella <03:42.08> ella <03:42.80> ae <03:43.46> ae <03:44.14> ae <03:44.84> ae <03:45.21> ae <03:45.53> ae
[03:45.80] <03:45.90> it's <03:46.24> rainin' <03:46.92> raining
[03:47.46] <03:47.93> oh <03:48.28> baby <03:48.65> it's <03:48.96> rainin' <03:49.64> raining
[03:50.24] <03:50.70> baby <03:51.36> come <03:51.75> here <03:52.43> to <03:52.75> me
[03:52.92] <03:53.31> come <03:53.93> here <03:54.12> to <03:54.40> me
[03:54.48] <03:54.52> it's <03:54.93> raining <03:55.53> raining
[03:56.52] <03:56.58> oh <03:56.90> baby <03:57.02> it's <03:57.21> raining <03:57.92> raining
[03:58.53] <03:58.59> you <03:58.90> can <03:59.30> always <03:59.46> come <03:59.59> here