Mid-level interview questionAsked at TikTok · interview write-up
Explain policy types in RL and modern policy-gradient methods such as TRPO, PPO and GAE, then compare linear attention and grouped-query attention.
Model answer
Policies are deterministic or stochastic, on- or off-policy. Policy gradients raise the log-probability of high-advantage actions. TRPO bounds each update with a KL trust region; PPO gets similar stability from a cheap clipped ratio. GAE weights TD errors exponentially, with lambda trading bias for variance.
Scroll sideways to see the whole diagram
In multi-head attention, each head projects its own queries, keys and values. Every key and value is cached for every token.
Text version
- In multi-head attention, each head projects its own queries, keys and values. Every key and value is cached for every token.
- Grouped-query attention keeps all the query heads but lets each group read one shared key-value head. Multi-query attention is the extreme: one K, V for all.
- The cache shrinks in proportion to the key-value heads, which is why most large open models use GQA.
Ainslie et al. (2023), GQA; Llama 2 70B's configuration: 64 query heads, 8 key-value heads (Touvron et al. 2023, Section 2.2).
Linear attention swaps softmax for a kernel map: linear cost, usually lower quality. GQA keeps exact attention and shares key-value heads across query groups, shrinking the KV cache.
Practise the rest
35 Training questions, 492 in all, each with a model answer. Free.
More from TikTok interview questions.
More Training interview questions
- Implement masked cross-entropy in NumPy that stays numerically stable, with label smoothing and a temperature.
- Build a pipeline that finds math documents in web crawls and scores their quality.
- Explain KL divergence for language models: which direction is which, what happens when supports don't match, and how it's used as a regulariser.
- When would you use mean squared error versus cross-entropy? And what is LoRA?
- Explain Adam and its bias correction, and why AdamW decouples weight decay.
- Predict whether an ad impression is clicked within 24 hours at a 0.7% positive rate, with high-cardinality IDs and delayed labels: how do you build and assess the model?