Preface ix
1 Introduction to Deep Learning 1
1.1 History of Deep Learning 1
1.2 A Probabilistic Theory of Deep Learning 4
1.3 Back Propagation and Regularization 14
1.4 Batch Normalization and VC Dimension 17
1.5 Neural Nets--Deep and Shallow Networks 18
1.6 Supervised and Semi-Supervised Learning 19
1.7 Deep Learning and Reinforcement Learning 21
2 Basics of TensorFlow 25
2.1 Tensors 25
2.2 Computational Graph and Session 27
2.3 Constants, Placeholders, and Variables 28
2.4 Creating Tensor 32
2.5 Working on Matrices 35
2.6 Activation Functions 36
2.7 Loss Functions 39
2.8 Common Loss Function 39
2.9 Optimizers 40
2.10 Metrics 41
3 Understanding and Working with Keras 45
3.1 Major Steps to Deep Learning Models 45
3.2 Load Data 47
3.3 Pre-Process Data 48
3.4 Define the Model 48
3.5 Compile the Model 49
3.6 Fit and Evaluate the Mode 51
3.7 Prediction 52
3.8 Save and Reload the Model 52
3.9 Additional Steps to Improve Keras Models 53
3.10 Keras with TensorFlow 55
4 Multilayer Perceptron 57
4.1 Artificial Neural Network 57
4.2 Single-Layer Perceptron 60
4.3 Multilayer Perceptron 61
4.4 Logistic Regression Model 61
4.5 Regression to MLP in TensorFlow 63
4.6 TensorFlow Steps to Build Models 63
4.7 Linear Regression in TensorFlow 63
4.8 Logistic Regression Mode in TensorFlow 67
4.9 Multilayer Perceptron in TensorFlow 69
4.10 Regression to MLP in Keras 72
4.11 Log-Linear Model 72
4.12 Keras Neural Network for Linear Regression 73
4.13 Keras Neural Network for Logistic Regression 73
4.14 MLPs on the Iris Data 75
4.15 MLPs on MNIST Data (Digit Classification) 76
4.16 MLPs on Randomly Generated Data 78
5 Convolutional Neural Networks in Tensorflow 81
5.1 CNN Architectures 81
5.2 Properties of CNN Representations 82
5.3 Convolution Layers, Pooling Layers - Strides - Padding and Fully Connected Layer 82
5.4 Why TensorFlow for CNN Models? 84
5.5 TensorFlow Code for Building an Image Classifier for MNIST Data 84
5.6 Using a High-Level API for Building CNN Models 88
5.7 CNN in Keras 88
5.8 Building an Image Classifier for MNIST Data in Keras 88
5.9 Building an Image Classifier with CIFAR-10 Data 89
5.10 Define the Model Architecture 90
5.11 Pre-Trained Models 91
6 RNN and LSTM 95
6.1 Concept of RNN 95
6.2 Concept of LSTM 96
6.3 Modes of LSTM 97
6.4 Sequence Prediction 98
6.5 Time-Series Forecasting with the LSTM Model 99
6.6 Speech to Text 100
6.7 Examples Using Each API 102
6.8 Text-to-Speech Conversion 105
6.9 Cognitive Service Providers 106
6.10 The Future of Speech Analytics 107
7 Developing Chatbot's Face Detection and Recognition 109
7.1 Why Chatbots? 109
7.2 Designs and Functions of Chatbot's 109
7.3 Steps for Building a Chatbot's 110
7.4 Best Practices of Chatbot Development 116
7.5 Face Detection 116
7.6 Face Recognition 117
7.7 Face Analysis 117
7.8 OpenCV--Detecting a Face, Recognition and Face Analysis 117
7.8.1 Face Detection 117
7.8.2 Face Recognition 120
7.9 Deep Learning-Based Face Recognition 124
7.10 Transfer Learning 127
7.11 API's 131
8 Advanced Deep Learning 133
8.1 Deep Convolutional Neural Networks (AlexNet) 133
8.2 Networks Using Blocks (VGG) 137
8.3 Network in Network (NiN) 140
8.4 Networks with Parallel Concatenations (GoogLeNet) 144
8.5 Residual Networks (ResNet) 148
8.6 Densely Connected Networks (DenseNet) 151
8.7 Gated Recurrent Units (GRU) 154
8.8 Long Short-Term Memory (LSTM) 156
8.9 Deep Recurrent Neural Networks (D-RNN) 158
8.10 Bidirectional Recurrent Neural Networks (Bi-RNN) 159
8.11 Machine Translation and the Dataset 160
8.12 Sequence to Sequence Learning 161
9 Enhanced Convolutional Neural Network 167
9.1 Introduction 167
9.2 Deep Learning-Based Architecture for Absence Seizure Detection 178
9.3 EEG Signal Pre-Processing Strategy and Channel Selection 180
9.4 Input Formulation and Augmentation of EEG Signal for Deep Learning Model 188
9.5 Deep Learning Based Feature Extraction and Classification 196
9.6 Performance Analysis 200
9.7 Summary 201
10 Conclusion 205
10.1 Introduction 205
10.2 Future Research Direction and Prospects 205
10.3 Research Challenges in Deep Learning 210
10.4 Practical Deep Learning Case Studies 210
10.4.1 Medicine: Epilepsy Seizure Onset Prediction 219
10.4.2 Using Data from Test Drills to Predict where to Drill for Oil 232
10.5 Summary 235
References 235
Index 239
