[Paper Review] Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
Paper Link Peri-LayerNorm: A Third Option Beyond Post-LN and Pre-LN TL;DR By simply adding another LayerNorm right after the residual …
22 minute
Gradient Explosion
All posts under tag "Gradient Explosion"
1 posts total
Sorted by date