Mid-level interview questionAsked at Figma · interview write-up
Before coding a BERT keyphrase span-extraction task, what would you clarify about the interface, tokenizer offsets, valid spans and the outputs?
Model answer
Interface: one keyphrase per text or several, overlapping or not, labelled as offsets or strings?
Tokenizer: which vocabulary, cased or not. Map character labels to token start and end with the fast tokenizer's offset mapping, and decide what happens when a label cuts through a subword. Texts over 512 tokens need sliding windows.
Valid spans: end not before start, a maximum length, a no-answer case. Outputs: top spans as character offsets with scores.
Practise the rest
11 Tokenization questions, 492 in all, each with a model answer. Free.
More from Figma interview questions.
More Tokenization interview questions
- When you send a prompt to a language model, what does the model actually receive?
- What is a token, and why do most models use subword tokens rather than whole words or single characters?
- How does byte pair encoding learn a vocabulary, and how does it then encode a word it never saw in training?
- A colleague estimates next month's API bill by counting the words in a batch of English and Hindi support tickets and multiplying by the price per token. What's wrong with the estimate, and how would you do it instead?
- You're moving a product from a model that uses cl100k_base to one that uses o200k_base. Nothing crashes in staging. What could still be broken, and what goes into the migration plan?
- A ticket summariser crashes because tiktoken raises ValueError on a ticket that contains the literal end-of-text marker. A teammate's fix is to pass allowed_special='all'. What's your review comment?