Linear algebra flashcards: 50 definitions and theorems

All 50 cards

1. 1
   Dot product
   $\mathbf{u} \cdot \mathbf{v} = \displaystyle\sum_i u_i v_i$
2. 2
   Dot product and the angle between vectors
   $\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert \cos\theta$
3. 3
   Euclidean norm of a vector
   $\lVert \mathbf{v} \rVert = \sqrt{\mathbf{v} \cdot \mathbf{v}} = \sqrt{\textstyle\sum_i v_i^2}$
4. 4
   When are two vectors orthogonal?
   When their dot product is zero: $\mathbf{u} \cdot \mathbf{v} = 0$
5. 5
   Cross product $\mathbf{u} \times \mathbf{v}$ in $\mathbb{R}^3$
   $(u_2 v_3 - u_3 v_2,\; u_3 v_1 - u_1 v_3,\; u_1 v_2 - u_2 v_1)$

   It is perpendicular to both vectors, oriented by the right-hand rule.
6. 6
   Magnitude of the cross product
   $\lVert \mathbf{u} \times \mathbf{v} \rVert = \lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert \sin\theta$

   It equals the area of the parallelogram the two vectors span.
7. 7
   Cauchy–Schwarz inequality
   $\lvert \mathbf{u} \cdot \mathbf{v} \rvert \le \lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert$
8. 8
   Triangle inequality
   $\lVert \mathbf{u} + \mathbf{v} \rVert \le \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert$
9. 9
   Projection of $\mathbf{v}$ onto $\mathbf{u}$
   $\operatorname{proj}_{\mathbf{u}} \mathbf{v} = \dfrac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\,\mathbf{u}$
10. 10
    When is the product $AB$ defined?
    When $A$ is $m \times n$ and $B$ is $n \times p$; the result is $m \times p$
11. 11
    Entry $(i, j)$ of $AB$
    $(AB)_{ij} = \displaystyle\sum_k a_{ik}\,b_{kj}$
12. 12
    Is matrix multiplication commutative?
    No: in general $AB \neq BA$

    It is associative and distributive, though.
13. 13
    Identity matrix
    A square matrix with ones on the diagonal and zeros elsewhere: $AI = IA = A$
14. 14
    Transpose of a product
    $(AB)^T = B^T A^T$
15. 15
    Inverse of a product
    $(AB)^{-1} = B^{-1} A^{-1}$
16. 16
    Inverse of a $2 \times 2$ matrix
    $$\begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$$

    It exists only when $ad - bc \neq 0$.
17. 17
    Trace of a matrix
    The sum of the diagonal entries: $\operatorname{tr}(A) = \sum_i a_{ii}$

    $\operatorname{tr}(A + B) = \operatorname{tr}(A) + \operatorname{tr}(B)$
18. 18
    Cyclic property of the trace
    $\operatorname{tr}(AB) = \operatorname{tr}(BA)$
19. 19
    Determinant of a $2 \times 2$ matrix $\begin{pmatrix} a & b \\ c & d \end{pmatrix}$
    $ad - bc$
20. 20
    Determinant of $\begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix}$
    $a(ei - fh) - b(di - fg) + c(dh - eg)$

    Cofactor expansion along the first row.
21. 21
    $\det(AB)$
    $\det(A)\,\det(B)$
22. 22
    $\det(A^T)$
    $\det(A)$
23. 23
    $\det(cA)$ for an $n \times n$ matrix
    $c^n \det(A)$
24. 24
    Effect of swapping two rows on the determinant
    The sign flips

    Adding a multiple of one row to another leaves it unchanged; scaling a row by c scales it by c.
25. 25
    When is a square matrix invertible?
    Exactly when $\det(A) \neq 0$

    Equivalently: full rank, only the trivial solution to Ax = 0, and 0 is not an eigenvalue.
26. 26
    Span of a set of vectors
    The set of all their linear combinations
27. 27
    Linear independence
    $c_1 \mathbf{v}_1 + \dots + c_k \mathbf{v}_k = \mathbf{0}$ only when every $c_i = 0$
28. 28
    Basis
    A linearly independent set of vectors that spans the space
29. 29
    Dimension of a vector space
    The number of vectors in any basis
30. 30
    Null space of A
    All vectors $\mathbf{x}$ with $A\mathbf{x} = \mathbf{0}$
31. 31
    Column space of A
    The span of the columns of $A$: all $\mathbf{b}$ for which $A\mathbf{x} = \mathbf{b}$ has a solution
32. 32
    Rank of a matrix
    The dimension of its column space, which equals the dimension of its row space and the number of pivots
33. 33
    Rank–nullity theorem ($A$ is $m \times n$)
    $\operatorname{rank}(A) + \operatorname{nullity}(A) = n$

    n is the number of columns.
34. 34
    Elementary row operations
    Swap two rows, multiply a row by a nonzero scalar, add a multiple of one row to another
35. 35
    Gaussian elimination
    Row-reduce the augmented matrix to row echelon form, then solve by back-substitution
36. 36
    How many solutions can $A\mathbf{x} = \mathbf{b}$ have?
    None, exactly one, or infinitely many

    None if inconsistent; one if consistent with no free variables; infinitely many if there are free variables.
37. 37
    Least-squares normal equations
    $A^T A\,\hat{\mathbf{x}} = A^T \mathbf{b}$
38. 38
    Projection matrix onto the column space of $A$ (full column rank)
    $P = A (A^T A)^{-1} A^T$
39. 39
    Eigenvalue and eigenvector
    $A\mathbf{v} = \lambda \mathbf{v}$ with $\mathbf{v} \neq \mathbf{0}$
40. 40
    Characteristic equation
    $\det(A - \lambda I) = 0$

    Its roots are the eigenvalues.
41. 41
    Sum and product of the eigenvalues
    Sum $= \operatorname{tr}(A)$, product $= \det(A)$

    Counted with multiplicity, over the complex numbers.
42. 42
    Diagonalization
    $A = PDP^{-1}$, with eigenvectors as the columns of $P$ and eigenvalues on the diagonal of $D$

    Possible exactly when $A$ has $n$ linearly independent eigenvectors.
43. 43
    An $n \times n$ matrix with $n$ distinct eigenvalues is…
    Diagonalizable
44. 44
    Matrix powers via diagonalization
    $A^k = P D^k P^{-1}$
45. 45
    Orthogonal matrix
    $Q^T Q = I$, so $Q^{-1} = Q^T$

    Its columns are orthonormal; it preserves lengths and angles, and det Q = ±1.
46. 46
    Gram–Schmidt process
    Turns a basis into an orthonormal basis: subtract from each vector its projections onto the previous ones, then normalize
47. 47
    Spectral theorem (real symmetric matrices)
    A real symmetric matrix has real eigenvalues and orthogonal eigenvectors: $A = Q \Lambda Q^T$
48. 48
    Positive definite matrix
    A symmetric matrix with $\mathbf{x}^T A \mathbf{x} > 0$ for every $\mathbf{x} \neq \mathbf{0}$

    Equivalently, all its eigenvalues are positive.
49. 49
    Singular value decomposition (SVD)
    $A = U \Sigma V^T$, with $U$ and $V$ orthogonal and $\Sigma$ diagonal with nonnegative entries

    Every real matrix has one.
50. 50
    Singular values of $A$
    The square roots of the eigenvalues of $A^T A$