digest
Short Summary
Main recommendation: Use five prompt engineering practices to get the best performance from Claude: describe your task clearly, mark prompt parts with XML tags, give many examples, use Claude's long context, and let Claude think before answering. Alex, a prompt engineer at Anthropic, presents these as tips drawn from Anthropic's test-driven approach, where new prompts are run against benchmarks to measure performance. The tips are demonstrated with a personal-information removal task.
Key Points
- Describe the task explicitly: spell out what the target content means, such as naming email addresses and phone numbers as personal identifiable information (PII) to remove.
- Wrap distinct parts of the prompt in XML tags, since Claude has been fine-tuned to pay special attention to their structure.
- Provide many examples covering a wide range of cases, because more examples help Claude learn the task.
- Use the long context window, which the speaker says can hold up to 100,000 tokens, roughly 70,000 words.
- Ask Claude to reason in thinking tags before its final answer on complex questions, since researchers found this improves performance.
Details Worth Noting
- Concrete instructions:
- Step 1, describe the task: write clear, direct, specific instructions and define the terms, for example "email addresses and phone numbers" for PII removal.
- Step 2, use XML tags: wrap the text Claude must work on in opening and closing tags so its beginning and end are clear.
- Step 3, add examples: include examples of the desired output inside XML tags, using a wide range of cases.
- Step 4, use long context: put large amounts of material into the prompt when needed, up to about 100,000 tokens.
- Step 5, let Claude think: add thinking tags so Claude can jot down ideas before its final answer on complex questions.
- Practice without API access: prompt engineering can be practiced at claude.ai even without API access.
- Anthropic's approach: the speaker says each new prompt is run against a series of benchmarks to measure performance scientifically.
- Background: the speaker says Anthropic's paper "Red Teaming Language Models to Reduce Harms" inspired their interest in red teaming and writing jailbreaks, which led them into prompt engineering.
- Jailbreaks: the speaker describes jailbreaks as specific prompts written to circumvent the filters applied on top of language models, also called red teaming attacks or prompt exploits.
Bottom Line
Claude's output quality depends heavily on how the prompt is structured, with clear task description, XML tags, examples, and room for reasoning doing most of the work.
