Preface x
List of Abbreviations xiii
1 Introduction 1
1.1 Robot Programming Methods 2
1.2 Programming by Demonstration 3
1.3 Historical Overview of Robot PbD 4
1.4 PbD System Architecture 6
1.4.1 Learning Interfaces 8
1.4.1.1 Sensor-Based Techniques 10
1.4.2 Task Representation and Modeling 13
1.4.2.1 Symbolic Level 14
1.4.2.2 Trajectory Level 16
1.4.3 Task Analysis and Planning 18
1.4.3.1 Symbolic Level 18
1.4.3.2 Trajectory Level 19
1.4.4 Program Generation and Task Execution 20
1.5 Applications 21
1.6 Research Challenges 25
1.6.1 Extracting the Teacher’s Intention from Observations 26
1.6.2 Robust Learning from Observations 27
1.6.2.1 Robust Encoding of Demonstrated Motions 27
1.6.2.2 Robust Reproduction of PbD Plans 29
1.6.3 Metrics for Evaluation of Learned Skills 29
1.6.4 Correspondence Problem 30
1.6.5 Role of the Teacher in PbD 31
1.7 Summary 32
References 33
2 Task Perception 43
2.1 Optical Tracking Systems 43
2.2 Vision Cameras 44
2.3 Summary 46
References 46
3 Task Representation 49
3.1 Level of Abstraction 50
3.2 Probabilistic Learning 51
3.3 Data Scaling and Aligning 51
3.3.1 Linear Scaling 52
3.3.2 Dynamic Time Warping (DTW) 52
3.4 Summary 55
References 55
4 Task Modeling 57
4.1 Gaussian Mixture Model (GMM) 57
4.2 Hidden Markov Model (HMM) 59
4.2.1 Evaluation Problem 61
4.2.2 Decoding Problem 62
4.2.3 Training Problem 62
4.2.4 Continuous Observation Data 63
4.3 Conditional Random Fields (CRFs) 64
4.3.1 Linear Chain CRF 65
4.3.2 Training and Inference 66
4.4 Dynamic Motion Primitives (DMPs) 68
4.5 Summary 70
References 70
5 Task Planning 73
5.1 Gaussian Mixture Regression 73
5.2 Spline Regression 74
5.2.1 Extraction of Key Points as Trajectories Features 75
5.2.2 HMM-Based Modeling and Generalization 80
5.2.2.1 Related Work 80
5.2.2.2 Modeling 81
5.2.2.3 Generalization 83
5.2.2.4 Experiments 87
5.2.2.5 Comparison with Related Work 100
5.2.3 CRF Modeling and Generalization 107
5.2.3.1 Related Work 107
5.2.3.2 Feature Functions Formation 107
5.2.3.3 Trajectories Encoding and Generalization 109
5.2.3.4 Experiments 111
5.2.3.5 Comparisons with Related Work 115
5.3 Locally Weighted Regression 117
5.4 Gaussian Process Regression 121
5.5 Summary 122
References 123
6 Task Execution 129
6.1 Background and Related Work 129
6.2 Kinematic Robot Control 132
6.3 Vision-Based Trajectory Tracking Control 134
6.3.1 Image-Based Visual Servoing (IBVS) 134
6.3.2 Position-Based Visual Servoing (PBVS) 135
6.3.3 Advanced Visual Servoing Methods 141
6.4 Image-Based Task Planning 141
6.4.1 Image-Based Learning Environment 141
6.4.2 Task Planning 142
6.4.3 Second-Order Conic Optimization 143
6.4.4 Objective Function 144
6.4.5 Constraints 146
6.4.5.1 Image-Space Constraints 146
6.4.5.2 Cartesian Space Constraints 149
6.4.5.3 Robot Manipulator Constraints 150
6.4.6 Optimization Model 152
6.5 Robust Image-Based Tracking Control 156
6.5.1 Simulations 157
6.5.1.1 Simulation 1 158
6.5.1.2 Simulation 2 161
6.5.2 Experiments 162
6.5.2.1 Experiment 1 166
6.5.2.2 Experiment 2 173
6.5.2.3 Experiment 3 177
6.5.3 Robustness Analysis and Comparisons with Other Methods 177
6.6 Discussion 183
6.7 Summary 185
References 185
Index 189
