Citadel interview questions

First remove the randomness you control: greedy decoding or a fixed seed, and a pinned model, prompt and parameters. Even then, GPU serving isn't bitwise deterministic, because batch composition changes floating-point reduction order, so cache outputs for exact replay.

For the score, treat runs as samples. Measure the standard deviation across repeats in a pilot, then pick n so the standard error, sd over root n, is well below the difference you need to detect.

Variable length: pad and carry a mask so padding never enters pooling. Order stops mattering: encode each point on its own and pool with a symmetric function, the Deep Sets recipe. Missing features: fill with zero and append a per-feature missing flag, so absence itself is a signal.

import torch
import torch.nn as nn

class SetClassifier(nn.Module):
    def __init__(self, d=6, h=64, n_cls=2):
        super().__init__()
        self.phi = nn.Sequential(
            nn.Linear(2 * d, h), nn.ReLU(),
            nn.Linear(h, h), nn.ReLU())
        self.head = nn.Linear(h, n_cls)

    def forward(self, x, keep):
        # x: (B, T, 6), NaN where a feature is missing
        # keep: (B, T) bool, True for real points
        miss = torch.isnan(x).float()
        x = torch.nan_to_num(x, nan=0.0)
        z = self.phi(torch.cat([x, miss], -1))
        z = z * keep.unsqueeze(-1).float()
        n = keep.sum(1, keepdim=True).clamp(min=1)
        return self.head(z.sum(1) / n)

Interviewers check the masking and a permutation test: shuffled points must give the same logits.