Machine learning visualisations
All visualisationsLanguage models

How does a self-attention head turn query-key similarity into weights?

Loading visualisation

Step 1 of 3Ask what ‘it’ refers to

Start with the tired sentence and select ‘it’. Compare its three strongest targets.