![[Paper Review] Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin](https://cdn-uploads.huggingface.co/production/uploads/6317233cc92fd6fee317e030/yNP71PjobVvLDgJ0R0qV2.png)
[Paper Review] Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Paper One Narrative Forged by Massive Activations: Bridging Attention Sink and Compression ValleyTL;DRMassive activations in the residual …
34 min
Massive Activations
TunedLens
LLM Internals
Activation Dynamics
Paper Review