![[Paper Review] Massive Activations in Large Language Models](https://eric-mingjie.github.io/massive-activations/assets/main_teaser_final.png)
[Paper Review] Massive Activations in Large Language Models
Paper Link Massive Activations, Hidden Biases: A Reinterpretation of Self-Attention’s Secrets TL;DRJust 4–10 extreme scalar values (×10,000) …
21 min
Transformer
Interpretability
NeuralMechanisms
Massive Activations
Explicit Attention Bias