Software testing basics: 50 cards on TDD, mocks and the test pyramid

All 50 cards

1. 1
   The test pyramid
   Many fast unit tests at the base, fewer integration tests in the middle, and a few end-to-end tests at the top

   Tests get slower, costlier and more brittle toward the top, so most checks belong lower down. Popularized by Mike Cohn.
2. 2
   Unit test
   Tests a small piece of code, such as a function or a class, in isolation; it runs in milliseconds, without a network, database or file system
3. 3
   Integration test
   Checks that several components work together correctly, e.g. your data-access code against a real database, or two services talking over HTTP
4. 4
   End-to-end (E2E) test
   Exercises the whole running system the way a user would, e.g. driving a browser with Playwright or Cypress through sign-up and checkout

   The most realistic kind of test, but also the slowest and the most prone to flakiness.
5. 5
   The “ice cream cone” anti-pattern
   An upside-down test pyramid: mostly manual and end-to-end tests with few unit tests, which makes feedback slow and flaky
6. 6
   The testing trophy
   Kent C. Dodds’ alternative to the pyramid: static analysis at the base, then unit tests, a large layer of integration tests, and a few end-to-end tests on top

   Its motto, from Guillermo Rauch: “Write tests. Not too many. Mostly integration.”
7. 7
   Arrange, Act, Assert
   A common structure for a test: set up the data, perform the one action under test, then check the outcome

   python

   def test_coupon_reduces_total():
       cart = Cart([Item(price=100)])  # Arrange
       cart.apply_coupon("SAVE10")     # Act
       assert cart.total() == 90       # Assert

8. 8
   Black-box vs white-box testing
   Black-box tests are designed from the specification without looking at the code; white-box tests are designed from the code’s internal structure, e.g. to cover every branch
9. 9
   Smoke test
   A quick, shallow check that the most important things work at all (the app starts, the home page loads, a user can sign in) before deeper testing
10. 10
    Regression testing
    Re-running tests after a change to make sure existing behavior still works

    When you fix a bug, first add a test that reproduces it, so the same bug can’t silently come back.
11. 11
    Acceptance testing
    Checks that the system meets the business requirements or acceptance criteria, usually from the user’s point of view; in user acceptance testing (UAT), real users or the customer do the checking
12. 12
    The TDD cycle
    Red: write a failing test. Green: write just enough code to make it pass. Refactor: clean up while the tests stay green

    Test-driven development was popularized by Kent Beck.
13. 13
    Benefits of test-driven development
    You define the behavior before writing code, the design is pushed toward small, decoupled, testable units, and you end up with a test suite that makes refactoring safe
14. 14
    Behavior-driven development (BDD)
    Describing behavior as concrete examples in business-readable language, usually Given / When / Then scenarios that run as tests, e.g. with Cucumber

    Given a cart containing one book When the customer applies the coupon SAVE10 Then the total is 10% lower
15. 15
    Test double
    Any object that stands in for a real dependency in a test; the umbrella term for dummies, stubs, spies, mocks and fakes

    The term comes from Gerard Meszaros, by analogy with stunt doubles in films.
16. 16
    Dummy (test double)
    An object passed only to fill a parameter list and never actually used
17. 17
    Stub
    A double that returns canned answers, so the test controls what the code under test receives, e.g. an exchange-rate service that always returns 1.1
18. 18
    Spy
    A double, or a wrapper around a real object, that records how it was called so the test can check those calls afterward
19. 19
    Mock
    A double programmed with expectations about the calls it should receive; the test fails if the interactions don’t match

    In everyday speech “mock” means any double, and libraries such as Jest and unittest.mock create objects that can act as stubs, spies or mocks.
20. 20
    Fake
    A real, working implementation that takes a shortcut unsuitable for production, e.g. an in-memory repository instead of a database
21. 21
    Stubs vs mocks: state vs behavior verification
    With stubs you check the result or final state (state verification); with mocks you check that the right calls were made (behavior verification)

    Heavy mocking ties tests to implementation details, so harmless refactors break them. See Martin Fowler’s essay “Mocks Aren’t Stubs”.
22. 22
    What does this test use the Mock for?

    python

    from unittest.mock import Mock
    
    def test_checkout_charges_card():
        gateway = Mock()
        gateway.charge.return_value = {"status": "ok"}
        checkout(Cart(total=4200), gateway)
        gateway.charge.assert_called_once_with(4200)

    Both roles: return_value makes it a stub that feeds checkout a canned reply, and assert_called_once_with verifies the call, as a mock would
23. 23
    “Don’t mock what you don’t own”
    Instead of mocking a third-party library directly, wrap it in a small interface of your own, use doubles of that interface in unit tests, and cover the wrapper with integration tests

    A mock of code you don’t control encodes your assumptions about it, which may be wrong or go out of date.
24. 24
    Line, branch and function coverage
    The share of lines executed, of decision outcomes taken (both sides of every if), and of functions called while the tests run
25. 25
    The only test calls discount(100, member=True). Line and branch coverage?

    python

    def discount(price, member):
        if member:
            price = price * 0.9
        return price

    100% line coverage, but only 50% branch coverage: the path where member is false never runs
26. 26
    Why doesn’t 100% code coverage mean the code is well tested?
    Coverage only shows that code ran, not that its results were checked: a test without assertions still counts, and missing cases or edge inputs don’t show up

    When coverage becomes a target, people write tests to hit lines rather than to catch bugs (Goodhart’s law). Use it to find untested code, not as a quality score.
27. 27
    Mutation testing
    A tool makes many small changes (mutants) to your code, such as > to >= or + to -, and runs the tests against each one. A mutant that no test catches reveals a gap in the tests

    Tools: PIT (Java), Stryker (JavaScript, C#, Scala), mutmut (Python), cargo-mutants (Rust).
28. 28
    Killed vs survived mutants, and the mutation score
    A mutant is killed if at least one test fails, and survives if all tests still pass. The mutation score is the share of mutants killed

    Mutation testing is slow, since the suite runs once per mutant, and some mutants behave exactly like the original code, so no test can kill them.
29. 29
    Property-based testing
    Instead of hand-picked examples, you state a property that must hold for all inputs, and the framework checks it against hundreds of generated inputs

    It started with QuickCheck for Haskell; Hypothesis (Python) and fast-check (JavaScript) are popular today.
30. 30
    What kind of test is this?

    python

    from hypothesis import given, strategies as st
    
    @given(st.lists(st.integers()))
    def test_sorting_twice_changes_nothing(xs):
        assert sorted(sorted(xs)) == sorted(xs)

    A property-based test: Hypothesis generates many lists of integers and checks, for every one, that sorting is idempotent
31. 31
    Shrinking (property-based testing)
    After finding a failing input, the framework simplifies it step by step to the smallest input that still fails, e.g. from a 40-element list down to [0, -1]
32. 32
    Useful properties to test
    Round trips (decode(encode(x)) == x), invariants (sorted output is ordered and has the same length), idempotence (doing it twice equals doing it once), and agreement with a simpler reference implementation
33. 33
    Flaky test
    A test that sometimes passes and sometimes fails on the same code

    Flaky tests teach a team to ignore red builds, so fix or quarantine them quickly.
34. 34
    Common causes of flaky tests
    Fixed sleeps and timing assumptions, race conditions in async code, shared state or dependence on test order, and relying on the real clock, randomness or network
35. 35
    How do you stop a UI test from failing on a slow page load?
    Wait for a condition, such as the element becoming visible, with a timeout, instead of sleeping for a fixed time

    Playwright’s locators and web-first assertions wait automatically.
36. 36
    Test isolation
    Each test sets up and cleans up its own state, so it passes on its own, in any order and in parallel with others

    Common tools: a database transaction rolled back after each test, temporary directories, and no shared mutable globals.
37. 37
    How do you test code that depends on the current time?
    Control the clock: inject a clock object, or use fake timers such as jest.useFakeTimers() or Python’s freezegun

    Reading the real clock directly makes results depend on when the tests run, e.g. failing only around midnight or on leap days.
38. 38
    Contract testing
    Checks that an API consumer and its provider agree on requests and responses by testing each side separately against a shared contract, without deploying both together
39. 39
    Consumer-driven contract testing (e.g. Pact)
    The consumer’s tests record the requests it makes and the responses it needs into a contract; the provider’s CI replays that contract against the real provider and fails on breaking changes
40. 40
    Test fixture
    The fixed setup a test needs, such as objects, data or database state, often shared through hooks like beforeEach or pytest fixtures
41. 41
    Parameterized test
    One test function that runs with several sets of inputs and expected outputs

    python

    import pytest
    
    @pytest.mark.parametrize("a, b, expected", [(1, 2, 3), (0, 0, 0), (-1, 1, 0)])
    def test_add(a, b, expected):
        assert add(a, b) == expected

42. 42
    Boundary value analysis
    Testing at and just around the edges of valid ranges, where off-by-one bugs live: for a valid range of 1–100, try 0, 1, 2, 99, 100 and 101
43. 43
    Equivalence partitioning
    Dividing inputs into groups the code should treat the same way, such as negative, zero and positive, and testing one representative of each group instead of every value
44. 44
    Snapshot testing
    Saving the output of a component or function and failing whenever it changes later

    Cheap to write, but snapshot changes are easy to approve without reading; keep snapshots small.
45. 45
    Characterization tests
    Tests that pin down what legacy code currently does, right or wrong, so you can refactor it safely

    Described by Michael Feathers in “Working Effectively with Legacy Code”.
46. 46
    The FIRST properties of good unit tests
    Fast, Independent, Repeatable, Self-validating (they pass or fail with no manual checking), Timely (written along with the code)
47. 47
    Why test behavior rather than implementation details?
    Tests that check observable results through the public interface survive refactoring; tests of private methods or internal calls break whenever the code is reorganized, even when it still works
48. 48
    Load vs stress vs soak vs spike testing
    Load: the expected traffic. Stress: beyond capacity, to find the breaking point. Soak: normal load for hours or days, to find leaks. Spike: a sudden surge
49. 49
    Fuzz testing
    Feeding huge numbers of random or malformed inputs to a program to find crashes, hangs and security bugs

    Coverage-guided fuzzers such as AFL++ and libFuzzer keep mutating the inputs that reach new code paths. Go has fuzzing built into go test.
50. 50
    Functional vs non-functional testing
    Functional tests check what the system does (its features and rules); non-functional tests check how well it does it: performance, security, usability, reliability