Quantization

4 pages

The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction

Paper Waking the MoE on SSD: Serving a 35B MoE at 20 tok/s on a 24 GB Desktop with Edge0One-line summary (TL;DR)Even at 4-bit, a 35B-class …

September 18, 2026

Mixture of Experts Quantization Efficient Inference LLM Inference Hardware Architecture Open-Source LLM

How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus

Paper Is “Lossless” Speculative Decoding Really Lossless? — The Numerical-Precision Trap Exposed by Reproducing OrthrusTL;DR — …

September 16, 2026

Speculative Decoding Numerical Precision Quantization LLM Inference Efficient Inference Benchmark Evaluation

SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

Paper SQS: Fusing Pruning and Quantization into One Bayesian Learning — Where Spike-and-Slab Meets GMMTL;DR — Doing pruning and low-bit …

September 13, 2026

Machine Learning Large Language Models Transformer Efficient Inference Model Compression Quantization

Miles v0.1: Production-Level Post-Training

Paper Miles v0.1: A “verified, clean, and scalable” full-stack system for frontier RL post-trainingTL;DR: Miles is a full-stack …

September 10, 2026

Reinforcement Learning Efficient Training Distributed Computing Mixture of Experts Quantization Large Language Models