Prompt engineering: 50 cards on LLMs, prompting techniques and RAG

All 50 cards

1. 1
   How does a large language model (LLM) generate text?
   It predicts a probability distribution over the next token, samples one, appends it to the input and repeats until it emits a stop token or hits a length limit

   Most LLMs are transformer networks trained on huge text corpora, then tuned to follow instructions.
2. 2
   Token
   The unit of text a model reads and writes: a whole word, part of a word, a punctuation mark or whitespace

   Each model has its own tokenizer, so the same text can be a different number of tokens on different models.
3. 3
   Rule of thumb: how many tokens is English text?
   About 4 characters, or ¾ of a word, per token, so 100 tokens ≈ 75 words

   Code and most non-English languages use more tokens for the same content.
4. 4
   Why do token counts matter?
   Context limits, pricing and latency are all measured in tokens; output tokens usually cost more than input tokens and take longer to generate
5. 5
   Context window
   The maximum number of tokens a model can handle in one request: system prompt, conversation history, documents, tool results and the generated output together

   Current models range from tens of thousands to over a million tokens.
6. 6
   What happens when a conversation outgrows the context window?
   Older content has to be dropped, truncated or summarized, or the request fails. The model can’t use anything that isn’t in the window
7. 7
   Does a chat API remember your previous requests?
   Usually not: the model itself is stateless, so you resend the relevant conversation history with every request

   Chat apps and some APIs store history for you, but it still has to fit in the context window.
8. 8
   Knowledge cutoff
   The date the model’s training data ends; it knows nothing later unless you put the information in the prompt or give it a search tool
9. 9
   Temperature
   Rescales the next-token probabilities before sampling: low values make output focused and repeatable, high values make it more varied. With logits $z_i$ and temperature $T$: $p_i = \dfrac{e^{z_i/T}}{\sum_j e^{z_j/T}}$

   Even temperature 0 isn’t guaranteed to be perfectly deterministic.
10. 10
    Temperature for data extraction vs brainstorming
    Low (around 0 to 0.3) for extraction, classification and code; higher (around 0.7 to 1) for brainstorming and creative writing

    Ranges differ by provider, e.g. 0–2 for OpenAI and 0–1 for Anthropic. Some reasoning models ignore or fix the setting.
11. 11
    Top-p (nucleus sampling)
    Samples only from the smallest set of most likely tokens whose probabilities add up to p, such as 0.9

    Top-k is similar but keeps a fixed number k of tokens. A common recommendation is to tune temperature or top-p, not both.
12. 12
    What does a max output tokens limit do?
    Caps the length of the response; if it is reached, the output stops mid-sentence and the API reports a length or max-tokens stop reason

    Stop sequences are strings that end generation early when the model produces them.
13. 13
    System prompt vs user message
    The system prompt, set by the developer, defines the role, rules and context for the whole conversation; user messages carry the individual requests. Models are trained to give system instructions priority

    Some APIs call it the developer message.
14. 14
    Role (persona) prompting
    Telling the model who it is, such as “You are a senior tax accountant reviewing a small business’s expenses”, to set its expertise, vocabulary and tone
15. 15
    Make this prompt specific: “Write about our new app.”
    State the audience, goal, format, length, tone and key facts, e.g. “Write a 120-word announcement for existing customers about our budgeting app’s new shared wallets. Friendly tone, one call to action, no jargon.”

    A good test: would a smart new colleague with no context know exactly what to produce?
16. 16
    Why wrap instructions and data in delimiters such as XML tags?
    They separate instructions from content so the model knows what to act on and what to treat as data, and you can refer to sections by name

    xml

    <instructions>
    Summarize the article in two sentences for a busy executive.
    </instructions>
    <article>
    {{ARTICLE_TEXT}}
    </article>

17. 17
    Where should a long document go in a prompt?
    Usually first, with the question and instructions after it; for very long inputs, repeat the key instructions at the end

    Models tend to use information at the start and end of a long context better than in the middle (“lost in the middle”).
18. 18
    Why phrase instructions positively?
    Saying what to do works better than listing what not to do: “Write in plain prose paragraphs” beats “Don’t use markdown”, and explaining why helps the model generalize
19. 19
    How do you give the model an “out”?
    Explicitly allow an answer like “I don’t know” or “Not found in the document”, which reduces made-up answers when the information is missing
20. 20
    Zero-shot prompting
    Asking for a task with instructions only, without any examples
21. 21
    Few-shot prompting
    markdown

    Classify the sentiment as positive, negative or neutral.
    
    Review: Arrived quickly and works great.
    Sentiment: positive
    
    Review: Stopped charging after two days.
    Sentiment: negative
    
    Review: {{review}}
    Sentiment:

    Including a few input–output examples shows the model the task, the labels and the exact output format.
22. 22
    How should you choose few-shot examples?
    Make them diverse, representative and consistent in format, and include tricky edge cases; the model copies their style and quirks, including mistakes and length
23. 23
    Chain-of-thought prompting
    Asking the model to reason step by step before giving its final answer, which improves multi-step tasks like math, logic and planning

    Ask for the reasoning and the final answer in separate sections (or tags) so code can extract the answer.
24. 24
    Reasoning models
    Models trained to work through a hidden or summarized chain of thought before answering, such as OpenAI’s o-series, DeepSeek-R1 or Claude with extended thinking

    For them, a clear goal and constraints matter more than spelling out each step; many let you set a reasoning effort or thinking budget.
25. 25
    Prompt chaining
    Splitting a complex task into a sequence of prompts where each output feeds the next step, such as extract → analyze → draft → review; each step is easier to test and debug
26. 26
    Structured output
    Asking the model to answer in a machine-readable format, usually JSON matching a schema, so your code can parse the result reliably
27. 27
    JSON mode vs schema-enforced structured outputs
    JSON mode guarantees syntactically valid JSON but not its shape; structured outputs constrain generation to a JSON Schema you supply, so required fields and types are guaranteed

    Still validate the result: the values can be wrong even when the shape is right.
28. 28
    Prompt pattern for extracting fields as JSON
    Describe each field, say what to do when it is missing, and show the exact shape

    json

    {
      "company": "string",
      "invoice_date": "YYYY-MM-DD or null",
      "total": "number or null",
      "currency": "ISO 4217 code, e.g. EUR"
    }

29. 29
    Tool (function) calling
    You describe functions with a name, description and JSON Schema for the arguments; the model replies with a call request, your code runs the function and sends back the result for the model to use

    With client-side tools the model never executes anything itself. Some providers also run built-in tools, such as web search, on their side.
30. 30
    AI agent
    An LLM that uses tools in a loop: it decides on a step, calls a tool, reads the result and repeats until the task is done or a limit is reached
31. 31
    Hallucination
    Fluent, confident output that is false or unsupported, such as invented facts, quotes, citations, URLs or API functions
32. 32
    Why do LLMs hallucinate?
    They are trained to produce plausible text, not to check facts; gaps in training data, the knowledge cutoff and pressure to always give an answer all make it worse
33. 33
    How can you reduce hallucinations?
    Ground answers in supplied sources (RAG), ask for supporting quotes or citations, allow “I don’t know”, use a lower temperature for factual tasks, and verify important claims

    No technique eliminates hallucinations, so high-stakes outputs need checks.
34. 34
    Embedding
    A vector of numbers representing the meaning of a text, so that texts with similar meanings have nearby vectors

    Typical embeddings have a few hundred to a few thousand dimensions.
35. 35
    Cosine similarity
    $\cos\theta = \dfrac{a \cdot b}{\lVert a\rVert\,\lVert b\rVert}$: the angle-based similarity of two vectors, 1 for the same direction

    The usual way to compare embeddings.
36. 36
    Vector database
    A store for embeddings with fast (usually approximate) nearest-neighbor search, so you can find the chunks most similar to a query

    Examples include pgvector for PostgreSQL, Qdrant, Pinecone, Weaviate and Chroma.
37. 37
    Retrieval-augmented generation (RAG)
    At query time, retrieve relevant documents (often by embedding search) and put them in the prompt, so the model answers from current, specific sources instead of memory alone
38. 38
    The steps of a basic RAG pipeline
    Indexing: split documents into chunks, embed them, store the vectors. Querying: embed the question, retrieve the top-k similar chunks, optionally rerank them, add them to the prompt and generate an answer with citations
39. 39
    Chunking trade-offs in RAG
    Chunks that are too large dilute relevance and waste context; chunks that are too small lose surrounding meaning. Splitting on headings or paragraphs, with some overlap, is a common start
40. 40
    Hybrid search
    Combining keyword search (such as BM25) with vector search, which catches exact terms like product codes and names that embeddings can miss

    The two result lists are often merged with reciprocal rank fusion.
41. 41
    Reranking
    A second stage where a more accurate model, usually a cross-encoder, scores each retrieved chunk against the query and reorders them before they go into the prompt
42. 42
    RAG vs fine-tuning
    RAG supplies knowledge at query time, so it stays current and can cite sources; fine-tuning changes the model’s weights to teach a style, format or task, and is a poor way to add facts that change
43. 43
    Prompt injection
    Input that contains instructions which override the developer’s, such as “Ignore all previous instructions and…”

    It is first in the OWASP Top 10 for LLM Applications.
44. 44
    Indirect prompt injection
    Malicious instructions hidden in content the model processes, such as a web page, email, PDF or retrieved document, rather than typed by the user
45. 45
    How do you limit the damage from prompt injection?
    There is no complete fix, so treat model output as untrusted: give tools least privilege, require human approval for sensitive actions, mark untrusted content clearly, and filter inputs and outputs
46. 46
    Jailbreak vs prompt injection
    A jailbreak tries to get the model to ignore its safety training; prompt injection hijacks an application by smuggling in instructions that override the developer’s
47. 47
    Is it safe to put secrets in a system prompt?
    No. Users can often get the model to reveal its system prompt, so treat it as public and keep API keys and confidential data out of it
48. 48
    Prompt evaluation (evals)
    A test set of representative inputs with expected outputs or grading criteria; run every prompt or model change against it and compare scores instead of eyeballing a few examples
49. 49
    LLM-as-a-judge
    Using a model to grade outputs against a rubric. It scales well but has biases (position, length, preferring its own style), so check it against human ratings
50. 50
    Prompt caching
    Providers can cache a repeated prompt prefix, such as a long system prompt, tool definitions or a document, making later requests cheaper and faster

    Put stable content first and the parts that change last, so the prefix stays identical.