https://jaehun.me/en/tags/audio-generation/Recent content in Audio Generation on Jaehun's BlogHugoen-USThu, 17 Sep 2026 00:27:04 +0900https://jaehun.me/en/posts/paper-2609-12945/Thu, 17 Sep 2026 00:00:00 +0900https://jaehun.me/en/posts/paper-2609-12945/<p><a href="https://arxiv.org/abs/2609.12945"target="_blank" class="inline-flex items-center gap-1" >Paper<svg class="h-3 w-3 flex-shrink-0" id="external-link" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 3h6v6m-11 5L21 3m-3 10v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/></svg> </a></p> <h2 id="stepaudio-3-gen-unifying-speech-song-music-and-sound-effects-with-a-single-discrete-autoregressive-rvq-model">StepAudio 3 Gen: Unifying Speech, Song, Music, and Sound Effects with a Single Discrete Autoregressive (RVQ) Model<a href="#stepaudio-3-gen-unifying-speech-song-music-and-sound-effects-with-a-single-discrete-autoregressive-rvq-model" class="heading-anchor" aria-label="Link to this section"><svg class="h-4 w-4" aria-hidden="true" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"/><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"/></g></svg></a></h2><h2 id="tldr">TL;DR<a href="#tldr" class="heading-anchor" aria-label="Link to this section"><svg class="h-4 w-4" aria-hidden="true" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"/><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"/></g></svg></a></h2><p><strong>StepAudio 3 Gen</strong>, released by StepFun, is a fully discrete autoregressive generator in which a $16 \times 2048$ residual vector quantization (RVQ) tokenizer at a 12.5 Hz frame rate compresses all audio into a single shared discrete code space, a pretrained LLM predicts the first codebook $c_0$ <strong>along the time axis</strong>, and a lightweight predictor predicts the remaining 15 codebooks $c_1,\dots,c_{15}$ <strong>along the depth axis</strong>. Despite abandoning the diffusion/flow-based continuous generation paradigm, it achieves state-of-the-art on TTS and voice design (§1, §3, §7).</p>