System prompt
Also called system prompts, system message
The developer's instructions, sent as the first message with the system role. The model gets it in the same sequence as the user's text; it carries more weight only through position and training.
Interview questions about System prompt
- A code assistant sends a 2,000-token system prompt and writes long answers. Users complain answers take too long to finish. Will a prefix cache fix it?
Mostly not. A prefix cache removes the prefix's prefill, so it cuts TTFT and input cost, not decode.
- You send a chat API a system message and a user message. What does the model actually receive, and why might the system prompt carry more weight?
One token sequence. The chat template writes each role as marker tokens around its content; AI Engineering: system and user prompts are "processed the same…
- Design a platform that lets teams evaluate their AI agents and see how changing the system prompt or the model affects quality.
Treat each configuration, model plus system prompt plus tools plus settings, as a versioned artifact.
- A reviewer says: “The system prompt forbids following instructions in emails, so the forward tool is safe, and our red-team run had zero violations.” Do you accept that?
No. A prompt instruction lowers the chance; the book notes "there's no guarantee that a model will follow the instructions given", and OWASP says it's unclear…
- Users ask about events after the model's training cutoff. How should it recognise those questions and say so?
The model doesn't know today's date, so tell it: put the date and the training cutoff in the system prompt. Then train the behaviour.
- Design the routing and scheduling layer in front of an inference fleet that serves many tenants.
Three jobs. Admission: per-tenant quotas counted in tokens, not requests, with token buckets and priority classes so interactive traffic beats batch.
- What is system-prompt caching, and what does it actually save?
The provider keeps the processed prefix, the keys and values prefill computed for your system prompt, and reuses them when the next request starts with the…
- You have two hours to build a working chatbot on the OpenAI API. Walk me through the message schema, session state, context-window strategy, streaming, and handling rate limits and retries.
Messages are role and content pairs: system, user, assistant. Store each session's list plus a summary by session ID.