Junior interview questionAsked at TikTok · interview write-up
Turn a multimodal model that captions videos into a production service under tight compute and GPU memory limits.
Model answer
Cut work before cutting the model. Sample a few keyframes instead of every frame, and drop near-duplicates. Use a smaller or quantized model, cap length, and batch requests to keep the GPU busy. Captions rarely need to be instant, so use an async queue and cache per video. Check quality on held-out videos.
Practise the rest
12 Multimodal questions, 492 in all, each with a model answer. Free.
More from TikTok interview questions.
More Multimodal interview questions
- What has driven each generation of LLM improvements, how do multimodal LLMs work, and what role does normalisation play in LLMs?
- You need to deploy multimodal models under tight GPU and latency limits, and also support fast retrieval over video. How do you approach it?
- How does a vision encoder feed images into a multimodal LLM, and where are the bottlenecks?
- Design the training of a multimodal embedding model: architecture, supervision and losses, evaluation metrics and deployment.
- Design a classifier that assigns genres to games using text, images and tabular metadata, from label quality to serving and monitoring.
- For an offline audio detection system, outline the ML inference and labeling pipeline: feature extraction, keyword spotting, thresholding, calibration and human labeling.