Human-Robot Interaction Control Using Reinforcement Learning

Author Biographies xi

List of Figures xiii

List of Tables xvii

Preface xix

Part I Human-robot Interaction Control 1

1 Introduction 3

1.1 Human-Robot Interaction Control 3

1.2 Reinforcement Learning for Control 6

1.3 Structure of the Book 7

References 10

2 Environment Model of Human-Robot Interaction 17

2.1 Impedance and Admittance 17

2.2 Impedance Model for Human-Robot Interaction 21

2.3 Identification of Human-Robot Interaction Model 24

2.4 Conclusions 30

References 30

3 Model Based Human-Robot Interaction Control 33

3.1 Task Space Impedance/Admittance Control 33

3.2 Joint Space Impedance Control 36

3.3 Accuracy and Robustness 37

3.4 Simulations 39

3.5 Conclusions 42

References 44

4 Model Free Human-Robot Interaction Control 45

4.1 Task-Space Control Using Joint-Space Dynamics 45

4.2 Task-Space Control Using Task-Space Dynamics 52

4.3 Joint Space Control 53

4.4 Simulations 54

4.5 Experiments 55

4.6 Conclusions 68

References 71

5 Human-in-the-loop Control Using Euler Angles 73

5.1 Introduction 73

5.2 Joint-Space Control 74

5.3 Task-Space Control 79

5.4 Experiments 83

5.5 Conclusions 92

References 94

Part II Reinforcement Learning for Robot Interaction Control 97

6 Reinforcement Learning for Robot Position/Force Control 99

6.1 Introduction 99

6.2 Position/Force Control Using an Impedance Model 100

6.3 Reinforcement Learning Based Position/Force Control 103

6.4 Simulations and Experiments 110

6.5 Conclusions 117

References 117

7 Continuous-Time Reinforcement Learning for Force Control 119

7.1 Introduction 119

7.2 K-means Clustering for Reinforcement Learning 120

7.3 Position/Force Control Using Reinforcement Learning 124

7.4 Experiments 130

7.5 Conclusions 136

References 136

8 Robot Control in Worst-Case Uncertainty Using Reinforcement Learning 139

8.1 Introduction 139

8.2 Robust Control Using Discrete-Time Reinforcement Learning 141

8.3 Double Q-Learning with k-Nearest Neighbors 144

8.4 Robust Control Using Continuous-Time Reinforcement Learning 150

8.5 Simulations and Experiments: Discrete-Time Case 154

8.6 Simulations and Experiments: Continuous-Time Case 161

8.7 Conclusions 170

References 170

9 Redundant Robots Control Using Multi-Agent Reinforcement Learning 173

9.1 Introduction 173

9.2 Redundant Robot Control 175

9.3 Multi-Agent Reinforcement Learning for Redundant Robot Control 179

9.4 Simulations and experiments 183

9.5 Conclusions 187

References 189

10 Robot H₂ Neural Control Using Reinforcement Learning 193

10.1 Introduction 193

10.2 H₂ Neural Control Using Discrete-Time Reinforcement Learning 194

10.3 H₂ Neural Control in Continuous Time 207

10.4 Examples 219

10.5 Conclusion 229

References 229

11 Conclusions 233

A Robot Kinematics and Dynamics 235

A.1 Kinematics 235

A.2 Dynamics 237

A.3 Examples 240

References 246

B Reinforcement Learning for Control 247

B.1 Markov decision processes 247

B.2 Value functions 248

B.3 Iterations 250

B.4 TD learning 251

Reference 258

Index 259