Senior interview questionAsked at Meta · interview write-up
Design an image generation system end to end: data curation, diffusion versus autoregressive architectures, training objectives, compute planning and evaluation.
Model answer
Data: large sets of image-text pairs filtered for quality, safety, duplicates and licensing, recaptioned by a vision-language model, since detailed captions improve prompt following. Architecture: latent diffusion, with a VAE compressing images and a diffusion transformer or U-Net denoising latents conditioned on a text encoder. Autoregressive models over image tokens fit naturally with LLMs but generate token by token. Objective: noise prediction or flow matching. Plan compute with small-scale scaling runs before the big one. Evaluate with FID, CLIP score, prompt-alignment benchmarks and human preference studies.
Practise the rest
12 Multimodal questions, 492 in all, each with a model answer. Free.
More from Meta interview questions.
More Multimodal interview questions
- What has driven each generation of LLM improvements, how do multimodal LLMs work, and what role does normalisation play in LLMs?
- You need to deploy multimodal models under tight GPU and latency limits, and also support fast retrieval over video. How do you approach it?
- How does a vision encoder feed images into a multimodal LLM, and where are the bottlenecks?
- Design the training of a multimodal embedding model: architecture, supervision and losses, evaluation metrics and deployment.
- Turn a multimodal model that captions videos into a production service under tight compute and GPU memory limits.
- Design a classifier that assigns genres to games using text, images and tabular metadata, from label quality to serving and monitoring.