Senior interview questionAsked at Datadog · interview write-up
Implement focal loss for class-imbalanced binary classification from basic tensor operations, keeping it numerically stable in logit space.
Model answer
Focal loss scales cross-entropy by (1 − p_t) to the power gamma, so well-classified examples add little and training focuses on hard ones; alpha rebalances the classes. Work from logits throughout.
import torch
def focal_loss(z, y, alpha=0.25, gamma=2.0):
# z: logits, y: float 0/1 targets, same shape
# stable BCE: max(z,0) - z*y + log(1+exp(-|z|))
ce = (z.clamp(min=0) - z * y
+ torch.log1p(torch.exp(-z.abs())))
p_t = torch.exp(-ce)
a_t = alpha * y + (1 - alpha) * (1 - y)
return (a_t * (1 - p_t) ** gamma * ce).mean()
Interviewers check you never take the log of a sigmoid output, that p_t comes from exp of minus the loss, and that gamma 0 recovers alpha-weighted cross-entropy.
Practise the rest
35 Training questions, 492 in all, each with a model answer. Free.
More from Datadog interview questions.
More Training interview questions
- Implement masked cross-entropy in NumPy that stays numerically stable, with label smoothing and a temperature.
- Build a pipeline that finds math documents in web crawls and scores their quality.
- Explain KL divergence for language models: which direction is which, what happens when supports don't match, and how it's used as a regulariser.
- When would you use mean squared error versus cross-entropy? And what is LoRA?
- Explain Adam and its bias correction, and why AdamW decouples weight decay.
- Explain policy types in RL and modern policy-gradient methods such as TRPO, PPO and GAE, then compare linear attention and grouped-query attention.