Introduction to Linear Algebra (Hardcover, 6 Revised edition)

1 Vectors and Matrices 1
1.1 Vectors and Linear Combinations . 2
1.2 Lengths and Angles from Dot Products . 9
1.3 Matrices and Their Column Spaces . 18
1.4 Matrix Multiplication AB and CR . 27

2 Solving Linear Equations Ax=b 39
2.1 Elimination and Back Substitution . 40
2.2 Elimination Matrices and Inverse Matrices . 49
2.3 Matrix Computations and A=LU . 57
2.4 Permutations and Transposes . 64

3 The Four Fundamental Subspaces 75
3.1 Vector Spaces and Subspaces . 76
3.2 Computing the Nullspace by Elimination : A=CR . 84
3.3 The Complete Solutionto Ax=b . 95
3.4 Independence, Basis, and Dimension . 106
3.5 Dimensions of the Four Subspaces . 120

4 Orthogonality 135
4.1 Orthogonality of Vectors and Subspaces . 136
4.2 Projections onto Lines and Subspaces . 143
4.3 Least Squares Approximations . 155
4.4 Orthonormal Bases and Gram-Schmidt . 168
4.5 The Pseudoinverse of a Matrix . 182

5 Determinants 191
5.1 3 by 3 Determinants and Cofactors . 192
5.2 Computing and Using Determinants . 198
5.3 Areas and Volumes by Determinants . 204

6 Eigenvalues and Eigenvectors 209
6.1 Introduction to Eigenvalues : Ax=λx . 210
6.2 Diagonalizing a Matrix . 225
6.3 Symmetric Positive Definite Matrices . 239
6.4 Complex Numbers and Vectors and Matrices . 255
6.5 Solving Linear Differential Equations . 263

7 The Singular Value Decomposition (SVD) 286
7.1 Singular Values and Singular Vectors. . 287
7.2 Image Processing by Linear Algebra . 297
7.3 Principal Component Analysis (PCAbytheSVD) . 302

8 Linear Transformations 308
8.1 The Idea of a Linear Transformation . 309
8.2 The Matrix of a Linear Transformation . 318
8.3 The Search for a GoodBasis . 327

9 Linear Algebra in Optimization 335
9.1 Minimizing a Multivariable Function . 336
9.2 Backpropagation and Stochastic Gradient Descent . 346
9.3 Constraints, Lagrange Multipliers, Minimum Norms . 355
9.4 Linear Programming, Game Theory, and Duality . 364

10 Learning from Data 370
10.1 Piecewise Linear Learning Functions. . 372
10.2 Creating and Experimenting . 381
10.3 Mean, Variance, and Covariance . 386

Appendix 1 The Ranks of AB and A+B 400
Appendix 2 Matrix Factorizations 401
Appendix 3 Counting Parameters in the Basic Factorizations 403
Appendix 4 Codes and Algorithms for Numerical Linear Algebra 404
Appendix 5 The Jordan Form of a Square Matrix 405
Appendix 6 Tensors 406
Appendix 7 The Condition Number of a Matrix Problem 407
Appendix 8 Markov Matrices and Perron-Frobenius 408
Appendix 9 Elimination and Factorization 410
Appendix 10 Computer Graphics 413

Index of Equations 419
Index of Notations 422
Index 423