Query, key and value
Also called queries, keys and values
Three learned projections of each token's vector. A query is compared with every key to get attention weights; the weights then mix the values.
Interview questions about Query, key and value
- Walk me through scaled dot-product attention. What are queries, keys and values, and why divide by the square root of d?
Score each token's query against every visible key, divide by √d, softmax, and weight the values.
- In a recommender that uses cross-attention between a user's behavior sequence and a candidate item, what exactly are the query, key and value?
The candidate item is the query; the user's behaviour sequence supplies the keys and values.
- What is attention, exactly how are the queries, keys and values computed, and how does a KV cache speed up decoding?
Attention builds each token's output as a weighted mix of other tokens.
- Cover the fundamentals: self- versus cross-attention, autoregressive decoding and sampling, where RL fits, and how you evaluate the result.
Self-attention draws queries, keys and values from one sequence; cross-attention takes queries from the decoder and keys and values from another source, like…
- What are multi-head and grouped-query attention, and what does GQA do to the KV cache?
In multi-head attention every head has its own queries, keys and values. Grouped-query attention lets a group of query heads share one key-value head.