Mid-level interview questionAsked at ByteDance · interview write-up
What has driven each generation of LLM improvements, how do multimodal LLMs work, and what role does normalisation play in LLMs?
Model answer
Generations gained from more and better data at compute-optimal scale, instruction tuning and preference training, then architecture tweaks like RoPE, grouped-query attention, MoE and RL for reasoning.
Multimodal LLMs run a vision encoder over image patches and project its outputs into the LLM's embedding space as extra tokens.
Normalisation keeps activations in range: pre-norm placement stabilises deep stacks, and RMSNorm drops mean-centring to save compute.
Practise the rest
12 Multimodal questions, 492 in all, each with a model answer. Free.
More from ByteDance interview questions.
More Multimodal interview questions
- You need to deploy multimodal models under tight GPU and latency limits, and also support fast retrieval over video. How do you approach it?
- How does a vision encoder feed images into a multimodal LLM, and where are the bottlenecks?
- Design the training of a multimodal embedding model: architecture, supervision and losses, evaluation metrics and deployment.
- Turn a multimodal model that captions videos into a production service under tight compute and GPU memory limits.
- Design a classifier that assigns genres to games using text, images and tabular metadata, from label quality to serving and monitoring.
- For an offline audio detection system, outline the ML inference and labeling pipeline: feature extraction, keyword spotting, thresholding, calibration and human labeling.