Netflix interview questions

Subword tokenizers keep common words whole and split rare ones into frequent pieces: no unknown words, short sequences. SentencePiece trains on raw text, spaces included, so it needs no language-specific pre-splitting.

Variants: encoder-only like BERT for understanding, decoder-only like GPT for generation, and encoder-decoder like T5 for input-to-output tasks such as translation.

Score each token's query against every visible key, divide by √d, softmax, and weight the values. The query is what a token looks for, the key what it offers, the value what it hands over. Dot products grow with d, and large scores push softmax onto one token, where gradients get tiny.

Self-attention: each token scores its query against every key, softmaxes, and sums the values by weight.

LoRA: freeze the weights, train a low-rank update.

Adam tracks running averages of each parameter's gradient and squared gradient, giving per-parameter step sizes: the transformer default. SGD uses one global rate and less memory; it often generalises well in vision.

ViT cuts an image into patches, embeds each as a token, and runs a standard transformer encoder.

It stores every processed token's keys and values, per layer and head, so each decode step computes only the new token's.

It's exact because of the causal mask: earlier tokens never depend on later ones, and greedy output was identical with and without the cache. After a 960-token prompt on our test machine, a step took 9.6 ms with the cache against 228.8 ms without, about 24 times faster.

Attention builds each token's output as a weighted mix of other tokens. Each hidden vector is multiplied by three learned matrices to give a query, key and value. Scores are query-key dot products divided by the root of the head size, masked causally in a decoder, softmaxed, then used to average the values.

In decoding, past tokens' keys and values never change, so the KV cache stores them. Each step projects only the new token and attends over the cache, instead of reprocessing the prefix, paying in memory that grows with batch times context.

MSE for regression, cross-entropy for classification. Cross-entropy is the negative log-likelihood of the right class; with softmax its gradient is just predicted minus true, so confident mistakes get big updates, while MSE on probabilities gives them small ones.

LoRA freezes the weights and trains a small low-rank update, so fine-tuning is far cheaper.

Jobs enter a queue with a priority and a team quota, with fair share across teams. Distributed training needs gang scheduling, all GPUs at once or none, and topology-aware placement on NVLink-connected GPUs in the same node or rack.

Batch inference is the filler: preemptible, sharded into small tasks, backfilling idle GPUs. Preemption relies on checkpointing, and so does recovery: health checks spot failed GPUs, and jobs restart from their last checkpoint. Track utilisation, queue wait per priority and fairness, and give users a clear view of why a job is waiting.