Preface xv
Acknowledgments xix
About the Author xxi
1 Introduction to Reinforcement and Systemic Machine Learning
1
1.1. Introduction 1
1.2. Supervised, Unsupervised, and Semisupervised Machine
Learning 2
1.3. Traditional Learning Methods and History of Machine
Learning 4
1.4. What Is Machine Learning? 7
1.5. Machine-Learning Problem 8
1.6. Learning Paradigms 9
1.7. Machine-Learning Techniques and Paradigms 12
1.8. What Is Reinforcement Learning? 14
1.9. Reinforcement Function and Environment Function 16
1.10. Need of Reinforcement Learning 17
1.11. Reinforcement Learning and Machine Intelligence 17
1.12. What Is Systemic Learning? 18
1.13. What Is Systemic Machine Learning? 18
1.14. Challenges in Systemic Machine Learning 19
1.15. Reinforcement Machine Learning and Systemic Machine
Learning 19
1.16. Case Study Problem Detection in a Vehicle 20
1.17. Summary 20
2 Fundamentals of Whole-System, Systemic, and
Multiperspective Machine Learning 23
2.1. Introduction 23
2.2. What Is Systemic Machine Learning? 27
2.3. Generalized Systemic Machine-Learning Framework 30
2.4. Multiperspective Decision Making and Multiperspective
Learning 33
2.5. Dynamic and Interactive Decision Making 43
2.6. The Systemic Learning Framework 47
2.7. System Analysis 52
2.8. Case Study: Need of Systemic Learning in the Hospitality
Industry 54
2.9. Summary 55
3 Reinforcement Learning 57
3.1. Introduction 57
3.2. Learning Agents 60
3.3. Returns and Reward Calculations 62
3.4. Reinforcement Learning and Adaptive Control 63
3.5. Dynamic Systems 66
3.6. Reinforcement Learning and Control 68
3.7. Markov Property and Markov Decision Process 68
3.8. Value Functions 69
3.8.1. Action and Value 70
3.9. Learning an Optimal Policy (Model-Based and Model-Free
Methods) 70
3.10. Dynamic Programming 71
3.11. Adaptive Dynamic Programming 71
3.12. Example: Reinforcement Learning for Boxing Trainer 75
3.13. Summary 75
4 Systemic Machine Learning and Model 77
4.1. Introduction 77
4.2. A Framework for Systemic Learning 78
4.3. Capturing the Systemic View 86
4.4. Mathematical Representation of System Interactions 89
4.5. Impact Function 91
4.6. Decision-Impact Analysis 91
4.7. Summary 97
5 Inference and Information Integration 99
5.1. Introduction 99
5.2. Inference Mechanisms and Need 101
5.3. Integration of Context and Inference 107
5.4. Statistical Inference and Induction 111
5.5. Pure Likelihood Approach 112
5.6. Bayesian Paradigm and Inference 113
5.7. Time-Based Inference 114
5.8. Inference to Build a System View 114
5.9. Summary 118
6 Adaptive Learning 119
6.1. Introduction 119
6.2. Adaptive Learning and Adaptive Systems 119
6.3. What Is Adaptive Machine Learning? 123
6.4. Adaptation and Learning Method Selection Based on Scenario
124
6.5. Systemic Learning and Adaptive Learning 127
6.6. Competitive Learning and Adaptive Learning 140
6.7. Examples 146
6.8. Summary 149
7 Multiperspective and Whole-System Learning 151
7.1. Introduction 151
7.2. Multiperspective Context Building 152
7.3. Multiperspective Decision Making and Multiperspective
Learning 154
7.4. Whole-System Learning and Multiperspective Approaches
164
7.5. Case Study Based on Multiperspective Approach 167
7.6. Limitations to a Multiperspective Approach 174
7.7. Summary 174
8 Incremental Learning and Knowledge Representation
177
8.1. Introduction 177
8.2. Why Incremental Learning? 178
8.3. Learning from What Is Already Learned. . . 180
8.4. Supervised Incremental Learning 191
8.5. Incremental Unsupervised Learning and Incremental
Clustering 191
8.6. Semisupervised Incremental Learning 196
8.7. Incremental and Systemic Learning 199
8.8. Incremental Closeness Value and Learning Method 200
8.9. Learning and Decision-Making Model 205
8.10. Incremental Classification Techniques 206
8.11. Case Study: Incremental Document Classification 207
8.12. Summary 208
9 Knowledge Augmentation: A Machine Learning Perspective
209
9.1. Introduction 209
9.2. Brief History and Related Work 211
9.3. Knowledge Augmentation and Knowledge Elicitation 215
9.4. Life Cycle of Knowledge 217
9.5. Incremental Knowledge Representation 222
9.6. Case-Based Learning and Learning with Reference to
Knowledge Loss 224
9.7. Knowledge Augmentation: Techniques and Methods 224
9.8. Heuristic Learning 228
9.9. Systemic Machine Learning and Knowledge Augmentation
229
9.10. Knowledge Augmentation in Complex Learning Scenarios
232
9.11. Case Studies 232
9.12. Summary 235
10 Building a Learning System 237
10.1. Introduction 237
10.2. Systemic Learning System 237
10.3. Algorithm Selection 242
10.4. Knowledge Representation 244
10.5. Designing a Learning System 245
10.6. Making System to Behave Intelligently 246
10.7. Example-Based Learning 246
10.8. Holistic Knowledge Framework and Use of Reinforcement
Learning 246
10.9. Intelligent Agents?Deployment and Knowledge
Acquisition and Reuse 250
10.10. Case-Based Learning: Human Emotion-Detection System
251
10.11. Holistic View in Complex Decision Problem 253
10.12. Knowledge Representation and Data Discovery 255
10.13. Components 258
10.14. Future of Learning Systems and Intelligent Systems
259
10.15. Summary 259
Appendix A: Statistical Learning Methods 261
Appendix B: Markov Processes 271
Index 281
