Senior interview questionAsked at OpenAI · interview write-up
Design a platform for fine-tuning LLMs on AWS, covering distributed training and security between tenants.
Model answer
A control plane accepts jobs, base model, dataset in the customer's S3, method and hyperparameters, validates them and schedules onto GPU pools. Default to LoRA: adapters train on few GPUs and many tenants share one served base model. Full fine-tunes use FSDP or ZeRO across EFA-linked nodes, checkpointing to S3 so spot interruptions resume.
Isolation is the hard part: a per-tenant KMS key for data and weights, jobs in tenant-scoped IAM roles with no cross-tenant network path, no shared caches holding tenant data, and every access audited.
Practise the rest
20 Fine-tuning & alignment questions, 492 in all, each with a model answer. Free.
More from OpenAI interview questions.
More Fine-tuning & alignment interview questions
- Prompting, RAG or fine-tuning: how do you choose?
- What is LoRA, and why is it so much cheaper than full fine-tuning?
- You're handed fine-tuning code whose loss behaves oddly. What do you check?
- Compare RLHF with a reward model to DPO.
- Debug a GRPO training loop. What is the probability ratio, and what goes wrong with it?
- How do you detect and prevent reward hacking during LLM post-training?