Soong and Lyricsmith logotype

Soong & Lyricsmith

Personal

Active

Two tools that form a single chain. Soong turns a music video into a lyrics video synced word by word to the voice. Lyricsmith transcribes a song's lyrics when Genius does not know them, and grades how reliable the result is.

View Project →

Project Genesis

Both tools came from a personal need: making karaoke-style lyrics videos quickly and easily, especially for lesser-known songs that have none on YouTube.

I started with transcription. These songs often have no lyrics online either, and I wanted to automate writing them so I could add them to Genius. That became Lyricsmith. It first separates the voice from the music with Demucs, which makes the transcription much cleaner, then cleans up the resulting text, removes repetitions and reflows it into verses. Since automatic transcription is never perfect, every song gets a grade from A to D, and doubtful passages are flagged instead of being silently kept: you know exactly how far to trust the result before publishing it.

I then moved on to the video side with Soong. You paste a link to a music video, Soong fetches the lyrics from Genius, then aligns every word on the singer's voice with forced alignment, instead of relying on approximate timestamps. It finally renders a blurred video with the lyrics burned in, line by line or word by word, in a fully adjustable style: font, size, outline, position, fade, blur strength. When the lyrics come from a transcription, you review them while listening to the track before anything is rendered.

They are two independent tools, each useful on its own, but they fit together naturally: when Genius does not know a song, Soong calls on Lyricsmith. I designed them to run on a personal machine, with a single 8 GB consumer graphics card and a CPU fallback, as a desktop app as well as from the command line.

Technical Choices

Lyrics
Genius first, Lyricsmith transcription as a fallback
Vocal separation
Demucs, before any transcription
Reliability
An A to D grade per song, doubtful passages flagged instead of kept
Sync
Forced alignment of every word on the voice
Rendering
FFmpeg, blurred video with burned-in lyrics, style set in one config.yaml
Interfaces
PySide6 desktop app, command line and Python library
Hardware
Designed for a single 8 GB consumer GPU, with a CPU fallback

Project Roadmap

  1. Lyricsmith

    Transcription chain: vocal separation, transcription, cleanup and quality grade.

    Completed
  2. Soong

    Lyrics video chain, with a desktop app and a command line.

    Completed
  3. Documentation

    Documentation sites of both tools migrated from MkDocs to Kiln.

    Completed