Mid-level interview questionAsked at Figma · interview write-up
Design an evaluation platform for a design-editing agent that handles both exact property changes and open-ended visual requests, combining hard assertions, rubrics and calibrated evaluators.
Model answer
Split tasks by verifiability. Exact requests, like setting a padding, get hard assertions on the document diff: the target property changed, and nothing else did.
Open-ended requests get a multimodal judge over before and after renders, scored per rubric criterion: intent followed, content preserved, layout intact. Calibrate each judge against designer labels and track agreement over time.
The platform: versioned files and instructions, sandboxed runs, stored traces, per-slice results, and a regression gate.
Practise the rest
8 AI judges questions, 492 in all, each with a model answer. Free.
More from Figma interview questions.
More AI judges interview questions
- What is LLM-as-a-judge, and why is it so widely used?
- How would you write the prompt for an LLM judge?
- Your judge gives the same reply different scores on different runs. What do you do?
- A vendor reports “faithfulness 4.2” and your pipeline reports “faithfulness 0.8”. Which system is more faithful?
- What biases do LLM judges have, and how do you mitigate them?
- Your judge agrees with human graders 80% of the time. Can you use it as a quality gate?