Scale AI interview questions

Two endpoints, one encoder: classification is a light head on the embedding. Interactive calls are synchronous, with dynamic batching to a latency target.

Batch is an async job: submit a file, get a job id, poll or take a webhook, read results from storage, trading latency for cost. Every response names the model version, because vectors from different versions aren't comparable.

Self-attention draws queries, keys and values from one sequence; cross-attention takes queries from the decoder and keys and values from another source, like an encoder's output.

Decoding is a loop: score the next token, pick one greedily or by temperature and top-p, append, repeat, with a KV cache. RL then tunes those sampled outputs against a reward, held near a reference model. Evaluate with perplexity, task benchmarks and human or judge preferences.

Template the prompt with the schema and one example, and use the API's JSON or structured-output mode if it has one. Parse the reply and validate it with Pydantic against the same schema. On failure, retry with the validation error, a few times at most, then fail loudly.

Use greedy coordinate gradient from Zou et al.: optimise one token suffix that makes a target completion likely across many prompts, which makes it universal. Token gradients propose swaps; exact loss picks the best.

import torch
import torch.nn.functional as F

def loss(model, E, prompt, oh, target):
    # E = model.transformer.wte.weight
    x = torch.cat([E[prompt], oh @ E, E[target]])
    out = model(inputs_embeds=x[None]).logits[0]
    n = len(target)
    return F.cross_entropy(out[-n - 1:-1], target)

def gcg_step(model, prompts, suf, target,
             k=64, tries=128):
    model.requires_grad_(False)  # grads on oh only
    E = model.transformer.wte.weight
    oh = F.one_hot(suf, E.shape[0]).to(E.dtype)
    oh.requires_grad_()
    sum(loss(model, E, p, oh, target)
        for p in prompts).backward()
    cand = (-oh.grad).topk(k, dim=1).indices
    best, best_l = suf, float("inf")
    for _ in range(tries):
        s = suf.clone()
        i = torch.randint(len(s), (1,)).item()
        s[i] = cand[i, torch.randint(k, (1,)).item()]
        with torch.no_grad():
            o = F.one_hot(s, E.shape[0]).to(E.dtype)
            l = sum(loss(model, E, p, o, target).item()
                    for p in prompts)
        if l < best_l:
            best, best_l = s, l
    return best

Evaluate safely: a local model, a benign target string, success rate on held-out prompts, and defences like perplexity filtering.

Before the model answers, retrieve relevant passages from your documents and put them in the prompt, so it answers from evidence in its context, not memory alone.

It cuts hallucination when retrieval finds the right passage and the prompt says to answer only from it, cite it, and admit when it isn't there.

Start with what the model actually sees: decode a batch back to text. Usual suspects: a chat template that differs from the base model's; labels shifted twice or not at all; a loss not masked to the answer tokens, so the model learns to predict prompts; padding counted in the loss; new special tokens missing from the tokenizer; a learning rate too high for the batch. Then overfit ten examples: if the loss won't approach zero, the bug is in the pipeline, not the data.

SFT imitates demonstrations: simple and stable, but can't exceed its data. RLHF trains a reward model on preferences and optimises against it with PPO: it can beat the demos, but is costly, unstable and prone to reward hacking. DPO-style methods learn from preference pairs directly: cheap and steady, but offline.

For checkable tasks like maths and code, RL with verifiable rewards, such as GRPO, skips the reward model. SFT usually comes first.