System Prompts: Design, Structure and Real-World Examples

The system prompt is the constitution of an AI application: persistent instructions that define identity, capabilities, rules and tone for every interaction. User messages come and go; the system prompt governs them all.

What belongs in a system prompt

A production system prompt typically defines, in order: identity (who the assistant is), capabilities and limits (what it can and cannot do), behavioral rules (safety, refusals, escalation), tone guidelines, and output format defaults. Anything that must hold across every conversation belongs here; anything task-specific belongs in the user message.

Structure it like a document, not a paragraph

Wall-of-text system prompts degrade as they grow. Use headed sections or XML tags so each rule has an address. This matters for maintenance: when a behavior needs changing, you edit one section instead of rereading a thousand words to find where the rule lives. The Prompt Expander scaffolds this structure for new prompts.

Rules that survive production

• Be explicit about refusals. “Politely decline requests outside billing topics and suggest contacting support” beats hoping the model infers boundaries.
• State priorities. When rules can conflict, say which wins: “If brevity conflicts with accuracy, prefer accuracy.”
• Constrain identity disclosure. Decide upfront what the assistant says about being an AI and about its instructions.
• Version everything. System prompts change; track them in git like the source code they are.

Template variables need discipline

System prompts commonly embed runtime variables: user names, dates, feature flags, retrieved context. Audit them with a variables extractor so your code supplies every placeholder, and remember that unfilled placeholders reach the model as literal braces, a confusing and surprisingly common production bug.

Never leak secrets into system prompts

System prompts frequently end up in logs, error reports and prompt-injection extractions. Treat their contents as semi-public: no API keys, no internal URLs you cannot expose, no customer data. Scan drafts with a privacy checker before deployment.

Start small, grow deliberately

The best system prompts start with ten lines and grow one tested rule at a time. Prompts that start at two thousand words were never tested rule by rule, and it shows.