DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
DeepSeekMoE 정리 – Dense 성능을 40 % FLOPs로 끌어낸 ‘세분화-전문가’ 트릭 한 줄 요약 (TL;DR) Fine-Grained Expert Segmentation (FGES) + Shared Experts (SEI) 로 FFN-MoE를 재설계한 DeepSeekMoE는 동일…