[eBook Code] Digital Signal Processing (DSP) with Python Programming (eBook Code, 1st)

Preface ix

Notations and Abbreviations xi

A Few Functions of Python® xiii

Chapter 1. Useful Maths  1

1.1. Basic concepts on probability  1

1.2. Conditional expectation  10

1.3. Projection theorem 11

1.3.1. Conditional expectation  14

1.4. Gaussianity 14

1.4.1. Gaussian random variable  14

1.4.2. Gaussian random vectors 15

1.4.3. Gaussian conditional distribution  16

1.5. Random variable transformation 18

1.5.1. General expression  18

1.5.2. Law of the sum of two random variables  19

1.5.3. δ-method  20

1.6. Fundamental theorems of statistics 22

1.7. A few probability distributions  24

Chapter 2. Statistical Inferences  29

2.1. First step: visualizing data  29

2.1.1. Scatter plot 29

2.1.2. Histogram/boxplot 30

2.1.3. Q-Q plot  32

2.2. Reduction of dataset dimensionality 34

2.2.1. PCA 34

2.2.2. LDA 36

2.3. Some vocabulary  40

2.3.1. Statistical inference  40

2.4. Statistical model  41

2.4.1. Notation  42

2.5. Hypothesis testing 43

2.5.1. Simple hypotheses 45

2.5.2. Generalized likelihood ratio test (GLRT)  50

2.5.3. χ2 goodness-of-fit test  57

2.6. Statistical estimation 58

2.6.1. General principles 58

2.6.2. Least squares method 62

2.6.3. Least squares method for the linear model 64

2.6.4. Method of moments  81

2.6.5. Maximum likelihood approach  84

2.6.6. Logistic regression  100

2.6.7. Non-parametric estimation of probability distribution 103

2.6.8. Bootstrap and others  107

Chapter 3. Inferences on HMM  113

3.1. Hidden Markov models (HMM) 113

3.2. Inferences on HMM  116

3.3. Filtering: general case 117

3.4. Gaussian linear case: Kalman algorithm  118

3.4.1. Kalman filter  118

3.4.2. RTS smoother 127

3.5. Discrete finite Markov case  129

3.5.1. Forward-backward formulas 130

3.5.2. Smoothing formula at one instant  133

3.5.3. Smoothing formula at two successive instants 134

3.5.4. HMM learning using the EM algorithm 135

3.5.5. The Viterbi algorithm 137

Chapter 4. Monte-Carlo Methods  141

4.1. Fundamental theorems  141

4.2. Stating the problem  141

4.3. Generating random variables 144

4.3.1. The cumulative function inversion method 144

4.3.2. The variable transformation method 147

4.3.3. Acceptance-rejection method 149

4.3.4. Sequential methods  151

4.4. Variance reduction 156

4.4.1. Importance sampling 156

4.4.2. Stratification  160

4.4.3. Antithetic variates 164

Chapter 5. Hints and Solutions 167

5.1. Useful maths  167

5.2. Statistical inferences  170

5.3. Inferences on HMM  226

5.4. Monte-Carlo methods 251

Bibliography 261

Index  263