Microsoft interview questions

By vocabulary, each device owns a range of rows, looks up the tokens it owns, writes zeros for the rest, and an all-reduce sum assembles full vectors. It pairs with a tied output head sharded the same way, but popular tokens can make one shard hot.

By hidden dimension, every device holds all rows but a slice of the width. Lookups are local, then an all-gather concatenates the slices. Both move about batch × tokens × width numbers; choose by what the next layer wants.

Embeddings: shortlist models on public retrieval benchmarks, then pick on your own queries; smaller dimensions or int8 vectors cut memory and latency.

Index: HNSW gives high recall at low latency but holds full vectors in memory; IVF with product quantisation compresses billions of vectors. Tune the recall-latency trade with ef or nprobe, then rerank a shortlist exactly.

Sharding: partition by user or tenant, since an assistant searches one person's content, which also enforces isolation; otherwise scatter-gather across shards and merge top-k. Add replicas for throughput.

Keep the k best partial sentences. Each step, extend every one with its likely next tokens, score by the sum of log-probabilities, and keep the top k. A sentence's probability is the exponent of that sum.

Greedy is beam width 1, so it misses sequences whose first token wasn't the best. Longer sentences collect more negative terms, so normalise by length when comparing finished beams.

Quality is task-specific: reference answers where they exist, a rubric judge, human spot checks. Hallucination against supplied context is faithfulness: split the answer into claims and check each is supported by the context.

import re
from dataclasses import dataclass, field

@dataclass
class Result:
    score: float
    unsupported: list = field(default_factory=list)

def claims(answer):
    parts = re.split(r"(?<=[.!?])\s+", answer)
    return [p.strip() for p in parts if p.strip()]

def faithfulness(answer, context, supports):
    # supports(context, claim) -> bool: an NLI
    # model or a judge prompt
    cs = claims(answer)
    if not cs:
        return Result(1.0)
    bad = [c for c in cs if not supports(context, c)]
    return Result(1 - len(bad) / len(cs), bad)

Interviewers look for claim-level scoring, a pluggable judge validated against human labels, and the empty case handled.

Don't ask the model for Word markup. Have it emit structured output, JSON with only allowed blocks such as heading with level, paragraph, list and table, each naming a permitted style, enforced by schema-constrained decoding.

A deterministic renderer then builds the document from the template's styles. Validate rules the schema can't express, like heading order, and retry with the error. Evaluate by diffing rendered styles against the spec.

Keep two stores per user: a small profile of stable facts and preferences, always in the prompt, and episode summaries retrieved by relevance.

After each conversation, a model extracts atomic facts with a timestamp and source; newer facts override conflicting ones, and duplicates merge. At query time, fit the profile plus the top retrieved memories into a fixed token budget.

The hard part is trust: users can see and delete memories, sensitive data needs consent and retention limits, stores are isolated per user, and remembered text is data, never instructions. Measure whether it helps.

Start from what success means for the product, then build an eval set from real traffic: sampled, labelled, covering known failure types. Score with code checks where possible and judge models validated against human labels elsewhere.

Every prompt, model or retrieval change runs the suite as a gate before release. In production, log inputs and outputs, collect user feedback, and canary or A/B test changes. Triage failures into categories, fix the biggest one, and add its examples to the eval set, so each round makes the next measurement sharper.

Index chunks with metadata and their source's access-control list, kept in sync. At query time, rewrite the query, run BM25 and dense retrieval with the permission filter inside both, fuse with reciprocal rank fusion, then a cross-encoder reranks the top few dozen. Measure each stage on labelled queries: recall at k for candidates, NDCG and MRR for the final order, then answer faithfulness and permission-leak tests.

SFT imitates demonstrations: simple and stable, but can't exceed its data. RLHF trains a reward model on preferences and optimises against it with PPO: it can beat the demos, but is costly, unstable and prone to reward hacking. DPO-style methods learn from preference pairs directly: cheap and steady, but offline.

For checkable tasks like maths and code, RL with verifiable rewards, such as GRPO, skips the reward model. SFT usually comes first.

Make every example executable: formulas parse, references point to real cells, and the expected output matches what a spreadsheet engine computes. Then deduplicate, check the test set doesn't overlap training, balance task types and sheet sizes, and have experts audit a sample. To decide on fine-tuning, evaluate the pretrained model with good prompts and examples on a held-out set first; fine-tune only if failures are systematic, like wrong function syntax.

Vocabulary sharding: each device holds a slice of rows and computes logits for its slice only. Softmax never needs the full vector: all-reduce the max, then the sum of exponentials. Hidden sharding: each device holds a slice of the hidden width and produces partial logits over the whole vocabulary, which must be all-reduced, a vocabulary-sized vector per token.

import torch
import torch.distributed as dist

def dist_topk(local, k, offset):
    # local: [B, V_shard] logits on this rank
    v, i = local.topk(k, dim=-1)
    i = i + offset  # to global vocabulary ids
    n = dist.get_world_size()
    vs = [torch.empty_like(v) for _ in range(n)]
    ix = [torch.empty_like(i) for _ in range(n)]
    dist.all_gather(vs, v)
    dist.all_gather(ix, i)
    v, i = torch.cat(vs, -1), torch.cat(ix, -1)
    top, pos = v.topk(k, dim=-1)
    return top, i.gather(-1, pos)

It's exact: every global top-k entry is in its own shard's top k.