Statistics flashcards: 50 formulas and concepts

All 50 cards

1. 1
   Sample mean
   $\bar{x} = \dfrac{1}{n}\displaystyle\sum_{i=1}^{n} x_i$
2. 2
   Median
   The middle value of the sorted data; with an even count, the average of the two middle values

   Unlike the mean, it is robust to outliers.
3. 3
   Mode
   The most frequent value in the data

   A dataset can have no mode, one mode, or several.
4. 4
   Range
   Maximum minus minimum
5. 5
   Interquartile range (IQR)
   $\mathrm{IQR} = Q_3 - Q_1$

   The spread of the middle 50% of the data.
6. 6
   Tukey’s rule for outliers
   Values below $Q_1 - 1.5\,\mathrm{IQR}$ or above $Q_3 + 1.5\,\mathrm{IQR}$
7. 7
   Population variance
   $\sigma^2 = \dfrac{1}{N}\displaystyle\sum_{i=1}^{N} (x_i - \mu)^2$
8. 8
   Sample variance
   $s^2 = \dfrac{1}{n - 1}\displaystyle\sum_{i=1}^{n} (x_i - \bar{x})^2$
9. 9
   Bessel’s correction
   Dividing by $n - 1$ instead of $n$ in the sample variance, which makes $s^2$ an unbiased estimator of $\sigma^2$
10. 10
    Standard deviation
    The square root of the variance

    It has the same units as the data.
11. 11
    Variance shortcut formula
    $\mathrm{Var}(X) = E[X^2] - (E[X])^2$
12. 12
    z-score
    $z = \dfrac{x - \mu}{\sigma}$

    How many standard deviations a value lies from the mean.
13. 13
    Standard normal distribution
    The normal distribution with mean $0$ and standard deviation $1$
14. 14
    Empirical rule (68–95–99.7)
    In a normal distribution, about 68%, 95% and 99.7% of values lie within 1, 2 and 3 standard deviations of the mean
15. 15
    Central limit theorem
    For independent samples from a population with finite variance, the distribution of the sample mean approaches a normal distribution as n grows, whatever the population’s shape
16. 16
    Standard error of the mean
    $\mathrm{SE} = \dfrac{\sigma}{\sqrt{n}}$

    In practice $\sigma$ is estimated with the sample standard deviation $s$.
17. 17
    Law of large numbers
    As the sample size grows, the sample mean converges to the expected value
18. 18
    95% confidence interval for a mean ($\sigma$ known)
    $\bar{x} \pm 1.96\,\dfrac{\sigma}{\sqrt{n}}$
19. 19
    What does a 95% confidence interval mean?
    If the sampling were repeated many times, about 95% of the intervals built this way would contain the true parameter
20. 20
    Null hypothesis $H_0$
    The default claim of no effect or no difference, which the test tries to find evidence against
21. 21
    Alternative hypothesis $H_1$
    The claim you accept if the data lead you to reject $H_0$
22. 22
    p-value
    The probability, assuming $H_0$ is true, of a result at least as extreme as the one observed
23. 23
    Significance level $\alpha$
    The threshold for rejecting $H_0$: reject when $p \le \alpha$. It is the probability of a type I error.

    $\alpha = 0.05$ is common.
24. 24
    Type I error
    Rejecting a true null hypothesis (a false positive); its probability is $\alpha$
25. 25
    Type II error
    Failing to reject a false null hypothesis (a false negative); its probability is $\beta$
26. 26
    Statistical power
    $1 - \beta$: the probability of rejecting $H_0$ when it is false

    Power grows with sample size and effect size.
27. 27
    One-sample t statistic
    $t = \dfrac{\bar{x} - \mu_0}{s/\sqrt{n}}$

    With $n - 1$ degrees of freedom.
28. 28
    When do you use a t-test instead of a z-test?
    When the population standard deviation is unknown and estimated by $s$, especially with small samples
29. 29
    Paired t-test
    Compares the means of two related measurements (e.g. before and after on the same subjects) by testing the mean of their differences
30. 30
    Chi-square test statistic
    $\chi^2 = \displaystyle\sum \frac{(O - E)^2}{E}$

    O observed counts, E expected counts.
31. 31
    Chi-square test of independence
    Tests whether two categorical variables are associated, using counts in a contingency table
32. 32
    Expected count in a contingency table cell
    $E = \dfrac{\text{row total} \times \text{column total}}{\text{grand total}}$
33. 33
    Correlation vs causation
    A correlation shows an association; it does not show that one variable causes the other

    Confounding variables or reverse causation can produce the same pattern.
34. 34
    Pearson correlation coefficient r
    A measure of linear association between $-1$ and $1$: its sign gives the direction, its size the strength
35. 35
    Formula for Pearson’s r
    $$r = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum (x_i - \bar{x})^2}\,\sqrt{\sum (y_i - \bar{y})^2}}$$
36. 36
    What does least-squares regression minimize?
    The sum of squared residuals
37. 37
    Residual
    $e_i = y_i - \hat{y}_i$: the observed value minus the predicted value
38. 38
    Least-squares slope and intercept
    $b_1 = r\,\dfrac{s_y}{s_x}$, $\;b_0 = \bar{y} - b_1\bar{x}$
39. 39
    Coefficient of determination $R^2$
    The proportion of the variance in $y$ explained by the model

    In simple linear regression, $R^2 = r^2$.
40. 40
    Conditional probability
    $P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$, for $P(B) > 0$
41. 41
    Bayes’ theorem
    $P(A \mid B) = \dfrac{P(B \mid A)\,P(A)}{P(B)}$
42. 42
    Independent events
    $P(A \cap B) = P(A)\,P(B)$

    Equivalently $P(A \mid B) = P(A)$.
43. 43
    Addition rule for probabilities
    $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
44. 44
    Mutually exclusive events
    Events that can’t both happen: $P(A \cap B) = 0$
45. 45
    Expected value of a discrete random variable
    $E[X] = \displaystyle\sum_x x\,P(X = x)$
46. 46
    Linearity of expectation
    $E[aX + bY] = a\,E[X] + b\,E[Y]$

    It holds even when X and Y are dependent.
47. 47
    Variance of $aX + b$
    $\mathrm{Var}(aX + b) = a^2\,\mathrm{Var}(X)$
48. 48
    Binomial probability mass function
    $P(X = k) = \dbinom{n}{k}\,p^k (1 - p)^{n - k}$

    n independent trials, each succeeding with probability p.
49. 49
    Mean and variance of a binomial distribution
    Mean $np$, variance $np(1 - p)$
50. 50
    Poisson probability mass function
    $P(X = k) = \dfrac{\lambda^k e^{-\lambda}}{k!}$

    Its mean and variance are both $\lambda$.