The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
논문 링크 SSD에 산 MoE를 깨우다: 24 GB 데스크톱에서 35B MoE를 20 tok/s로 서빙하는 Edge0한 줄 요약 (TL;DR)35B급 MoE는 4-bit로도 19.5 GB 를 차지해 24 GB 머신에 상주하지 못한다 (근거: §1). …
16분
Mixture of Experts
Quantization
Efficient Inference
LLM Inference
Hardware Architecture
Open-Source LLM
![[논문리뷰] KIMI K2: OPEN AGENTIC INTELLIGENCE](https://github.com/MoonshotAI/Kimi-K2/raw/main/figures/kimi-logo.png){kind=logo}