Autonomous Learning Systems: From Data Streams to Knowledge in Real-Time (Hardcover, New)

Forewords xi

Preface xix

About the Author xxiii

1 Introduction 1

1.1 Autonomous Systems 3

1.2 The Role of Machine Learning in Autonomous Systems 4

1.3 System Identification – an Abstract Model of the Real World 6

1.4 Online versus Offline Identification 9

1.5 Adaptive and Evolving Systems 10

1.6 Evolving or Evolutionary Systems 11

1.7 Supervised versus Unsupervised Learning 13

1.8 Structure of the Book 14

PART I FUNDAMENTALS

2 Fundamentals of Probability Theory 19

2.1 Randomness and Determinism 20

2.2 Frequentistic versus Belief-Based Approach 22

2.3 Probability Densities and Moments 23

2.4 Density Estimation – Kernel-Based Approach 26

2.5 Recursive Density Estimation (RDE) 28

2.6 Detecting Novelties/Anomalies/Outliers using RDE 32

2.7 Conclusions 36

3 Fundamentals of Machine Learning and Pattern Recognition 37

3.1 Preprocessing 37

3.2 Clustering 42

3.3 Classification 56

3.4 Conclusions 58

4 Fundamentals of Fuzzy Systems Theory 61

4.1 Fuzzy Sets 61

4.2 Fuzzy Systems, Fuzzy Rules 64

4.3 Fuzzy Systems with Nonparametric Antecedents (AnYa) 69

4.4 FRB (Offline) Classifiers 73

4.5 Neurofuzzy Systems 75

4.6 State Space Perspective 79

4.7 Conclusions 81

PART II METHODOLOGY OF AUTONOMOUS LEARNING SYSTEMS

5 Evolving System Structure from Streaming Data 85

5.1 Defining System Structure Based on Prior Knowledge 85

5.2 Data Space Partitioning 86

5.3 Normalisation and Standardisation of Streaming Data in an Evolving Environment 96

5.4 Autonomous Monitoring of the Structure Quality 98

5.5 Short- and Long-Term Focal Points and Submodels 104

5.6 Simplification and Interpretability Issues 105

5.7 Conclusions 107

6 Autonomous Learning Parameters of the Local Submodels 109

6.1 Learning Parameters of Local Submodels 110

6.2 Global versus Local Learning 111

6.3 Evolving Systems Structure Recursively 113

6.4 Learning Modes 116

6.5 Robustness to Outliers in Autonomous Learning 118

6.6 Conclusions 118

7 Autonomous Predictors, Estimators, Filters, Inferential Sensors 121

7.1 Predictors, Estimators, Filters – Problem Formulation 121

7.2 Nonlinear Regression 123

7.3 Time Series 124

7.4 Autonomous Learning Sensors 125

7.5 Conclusions 131

8 Autonomous Learning Classifiers 133

8.1 Classifying Data Streams 133

8.2 Why Adapt the Classifier Structure? 134

8.3 Architecture of Autonomous Classifiers of the Family AutoClassify 135

8.4 Learning AutoClassify from Streaming Data 139

8.5 Analysis of AutoClassify 140

8.6 Conclusions 140

9 Autonomous Learning Controllers 143

9.1 Indirect Adaptive Control Scheme 144

9.2 Evolving Inverse Plant Model from Online Streaming Data 145

9.3 Evolving Fuzzy Controller Structure from Online Streaming Data 147

9.4 Examples of Using AutoControl 148

9.5 Conclusions 153

10 Collaborative Autonomous Learning Systems 155

10.1 Distributed Intelligence Scenarios 155

10.2 Autonomous Collaborative Learning 157

10.3 Collaborative Autonomous Clustering, AutoCluster by a Team of ALSs 158

10.4 Collaborative Autonomous Predictors, Estimators, Filters and AutoSense by a Team of ALSs 159

10.5 Collaborative Autonomous Classifiers AutoClassify by a Team of ALSs 160

10.6 Superposition of Local Submodels 161

10.7 Conclusions 161

PART III APPLICATIONS OF ALS

11 Autonomous Learning Sensors for Chemical and Petrochemical Industries 165

11.1 Case Study 1: Quality of the Products in an Oil Refinery 165

11.2 Case Study 2: Polypropylene Manufacturing 172

11.3 Conclusions 178

12 Autonomous Learning Systems in Mobile Robotics 179

12.1 The Mobile Robot Pioneer 3DX 179

12.2 Autonomous Classifier for Landmark Recognition 180

12.3 Autonomous Leader Follower 193

12.4 Results Analysis 196

13 Autonomous Novelty Detection and Object Tracking in Video Streams 197

13.1 Problem Definition 197

13.2 Background Subtraction and KDE for Detecting Visual Novelties 198

13.3 Detecting Visual Novelties with the RDE Method 203

13.4 Object Identification in Image Frames Using RDE 204

13.5 Real-time Tracking in Video Streams Using ALS 206

13.6 Conclusions 209

14 Modelling Evolving User Behaviour with ALS 211

14.1 User Behaviour as an Evolving Phenomenon 211

14.2 Designing the User Behaviour Profile 212

14.3 Applying AutoClassify0 for Modelling Evolving User Behaviour 215

14.4 Case Studies 216

14.5 Conclusions 221

15 Epilogue 223

15.1 Conclusions 223

15.2 Open Problems 227

15.3 Future Directions 227

APPENDICES

Appendix A Mathematical Foundations 231

Appendix B Pseudocode of the Basic Algorithms 235

References 245

Glossary 259

Index 263