Large Language Models

8 pages

StepAudio 3 Gen Technical Report

Paper StepAudio 3 Gen: Unifying Speech, Song, Music, and Sound Effects with a Single Discrete Autoregressive (RVQ) ModelTL;DRStepAudio 3 …

September 17, 2026

Multimodal Learning Large Language Models Transformer Audio Generation Text-to-Speech Reinforcement Learning

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

Paper SAS: A Simple Attention Sparsification That Learns Context Ranking Directly Without DistillationTL;DR — In post-training attention …

September 15, 2026

KV Cache Long Context Efficient Inference Attention Optimization Large Language Models Transformer

SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

Paper SQS: Fusing Pruning and Quantization into One Bayesian Learning — Where Spike-and-Slab Meets GMMTL;DR — Doing pruning and low-bit …

September 13, 2026

Machine Learning Large Language Models Transformer Efficient Inference Model Compression Quantization

X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation

Paper X-AuT: Pruning Audio Encoders in Speech LLMs with Behavioral Probes and Cross-Scale DistillationTL;DRX-AuT is a framework that …

September 12, 2026

Efficient Inference Large Language Models Transformer Machine Learning Knowledge Distillation Model Compression

Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs

Paper Why Are Video LLMs Still So Expensive? — A Four-Stage Guide to Inference Efficiency MechanismsTL;DR — This paper classifies 125 …

September 11, 2026

Long Context KV Cache Multimodal Learning Vision-Language Model Efficient Inference Large Language Models

Miles v0.1: Production-Level Post-Training

Paper Miles v0.1: A “verified, clean, and scalable” full-stack system for frontier RL post-trainingTL;DR: Miles is a full-stack …

September 10, 2026

Reinforcement Learning Efficient Training Distributed Computing Mixture of Experts Quantization Large Language Models

[Paper Review] Peri-LN: Revisiting Normalization Layer in the Transformer Architecture

Paper Link Peri-LayerNorm: A Third Option Beyond Post-LN and Pre-LNTL;DRBy simply adding another LayerNorm right after the residual …

Large Language Models FP16 Training Empirical Evaluation Gradient Explosion Benchmark Evaluation

Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling

Paper Janus-Pro 7B: Dual-Encoder Multimodal LLM That Outsmarts Bigger ModelsOne-line summary (TL;DR)By fully separating the SigLIP …

DeepSeek Dual Encoder Multimodal Learning Vision-Language Models Large Language Models Efficient Training