Sierra interview questions

Assume the provider will be slow, down or wrong. Set timeouts on time to first token, retry with backoff and jitter only on safe errors, and use a circuit breaker to fail over to another provider or smaller model. Validate each output against a schema and policy checks before showing it; on failure, retry once, then use a scripted reply or human handoff. Keep state server-side so failover resumes cleanly.

def load_history(fetch_page):
    msgs, cursor = [], None
    while True:
        page = fetch_page(cursor)
        msgs.extend(page["messages"])
        cursor = page.get("next_cursor")
        if not cursor:
            return msgs

def compact(msgs, limit, count, summarize, keep=6):
    if sum(count(m) for m in msgs) <= limit:
        return msgs
    while True:
        cut = max(len(msgs) - keep, 0)
        note = {"role": "system",
                "content": summarize(msgs[:cut])}
        out = [note] + msgs[cut:]
        if sum(count(m) for m in out) <= limit:
            return out
        if keep == 0:
            raise ValueError("summary exceeds limit")
        keep //= 2

Follow the cursor until none comes back, retrying failed pages. Compact only when over the limit: summarise older turns into one note, keep the recent turns verbatim, and halve that window if it still doesn't fit. Count with the model's own tokenizer, and never separate a tool call from its result.