Asking word
Attention weights
The weights always add up to 100%. Greyed rows are later words, which the asking word is not allowed to see.
Part 1 of 10
AnatomyLesson 5 of 7
One attention head on a bench: a question, a label on every word, and shares that always add up to one.
The weights always add up to 100%. Greyed rows are later words, which the asking word is not allowed to see.
Part 1 of 10