Feed-Forward Network
lovely
woods
FFN
W
1
→
expand
σ
ReLU
→
W
2
Updated "
lovely
"
Updated "
woods
"
Fig. 7.
Each token embedding — lovely and woods — is fed through the same small FFN: expand with W₁, apply ReLU, then contract with W₂. Positions share weights; they do not mix.