Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?

A full softmax over millions of items is too costly, so each positive competes against sampled negatives. In-batch negatives appear in proportion to popularity, so popular items get over-penalised. The logQ correction subtracts each item's log sampling probability from its logit, which makes the sampled loss approximate the full softmax.

import torch
import torch.nn.functional as F

def in_batch_loss(u, items, log_q):
    # u, items: [B, d]; row i of items is user i's
    # positive; log_q: [B] log sampling probability
    logits = u @ items.T - log_q[None, :]
    labels = torch.arange(len(u), device=u.device)
    return F.cross_entropy(logits, labels)

Loss: softmax cross-entropy when exactly one item is right among many; per-item binary cross-entropy for multi-label targets or calibrated click probabilities.