Jailbreaking

Also called jailbreak, jailbreaks

Getting a model past its safety features. OWASP treats it as a form of prompt injection.

Interview questions about Jailbreaking

What's the difference between a jailbreak and a prompt injection?
A jailbreak gets past the model's safety features. Prompt injection adds instructions that make the app do something unintended: typed by the user (direct)…
Design an ML system that detects harmful content in both the prompts users send to an LLM and the responses it gives.
Run a classifier on both sides: on the prompt before the model, and on the response after, because a harmless prompt can still produce a harmful answer.