Introduction 1
About This Book 1
Foolish Assumptions 2
Icons Used in This Book 2
Beyond the Book 3
Where to Go from Here 4
Part 1: Getting to Know Tensorflow 5
Chapter 1: Introducing Machine Learning with TensorFlow 7
Understanding Machine Learning 7
The Development of Machine Learning 8
Statistical regression 9
Reverse engineering the brain 10
Steady progress 11
The computing revolution 12
The rise of big data and deep learning 12
Machine Learning Frameworks 13
Torch 14
Theano 14
Caffe 14
Keras 15
TensorFlow 15
Chapter 2: Getting Your Feet Wet 17
Installing TensorFlow 17
Python and pip/pip3 18
Installing on Mac OS 19
Installing on Linux 20
Installing on Windows 20
Exploring the TensorFlow Installation 21
Running Your First Application 22
Exploring the example code 23
Launching Hello TensorFlow! 23
Setting the Style 24
Chapter 3: Creating Tensors and Operations 27
Creating Tensors 27
Creating Tensors with Known Values 28
The constant function 30
zeros, ones, and fill 30
Creating sequences 31
Creating Tensors with Random Values 31
Transforming Tensors.33
Creating Operations 35
Basic math operations 35
Rounding and comparison 37
Exponents and logarithms 38
Vector and matrix operations 39
Putting Theory into Practice 42
Chapter 4: Executing Graphs in Sessions 45
Forming Graphs 46
Accessing graph data 47
Creating GraphDefs 49
Creating and Running Sessions 51
Creating sessions 51
Executing a session 52
Interactive sessions 53
Writing Messages to the Log 54
Visualizing Data with TensorBoard 56
Running TensorBoard 57
Generating summary data 57
Creating custom summaries 59
Writing summary data 59
Putting Theory into Practice 62
Chapter 5: Training 65
Training in TensorFlow 66
Formulating the Model 66
Looking at Variables 67
Creating variables 68
Initializing variables 68
Determining Loss 69
Minimizing Loss with Optimization 70
The Optimizer class 70
The GradientDescentOptimizer 71
The MomentumOptimizer 75
The AdagradOptimizer 76
The AdamOptimizer 77
Feeding Data into a Session 78
Creating placeholders 79
Defining the feed dictionary 79
Stochasticity 80
Monitoring Steps, Global Steps, and Epochs 80
Saving and Restoring Variables 82
Saving variables 82
Restoring variables 83
Working with SavedModels 84
Saving a SavedModel 85
Loading a SavedModel 86
Putting Theory into Practice 86
Visualizing the Training Process 89
Session Hooks 90
Creating a session hook 91
Creating a MonitoredSession 93
Putting theory into practice 94
Part 2: Implementing Machine Learning 97
Chapter 6: Analyzing Data with Statistical Regression 99
Analyzing Systems Using Regression 100
Linear Regression: Fitting Lines to Data 100
Polynomial Regression: Fitting Polynomials to Data 103
Binary Logistic Regression: Classifying Data into Two Categories 105
Setting up the problem 105
Defining models with the logistic function 106
Computing loss with maximum likelihood estimation 107
Putting theory into practice 108
Multinomial Logistic Regression: Classifying Data into Multiple Categories 110
The Modified National Institute of Science and Technology (MNIST) Dataset 110
Defining the model with the softmax function 113
Computing loss with cross entropy 114
Putting theory into practice 115
Chapter 7: Introducing Neural Networks and Deep Learning 117
From Neurons to Perceptrons 117
Neurons 118
Perceptrons 119
Improving the Model 121
Weights 121
Bias 122
Activation functions 123
Layers and Deep Learning 127
Layers 128
Deep learning 129
Training with Backpropagation 129
Implementing Deep Learning 131
Tuning the Neural Network 133
Input standardization 134
Weight initialization 135
Batch normalization 136
Regularization 139
Managing Variables with Scope 141
Variable scope 141
Retrieving variables from collections 142
Scopes for names and arguments 143
Improving the Deep Learning Process 143
Creating tuned layers 144
Putting theory into practice 145
Chapter 8: Classifying Images with Convolutional Neural Networks (CNNs) 149
Filtering Images 149
Convolution 150
Averaging Filter 151
Filters and features 152
Feature detection analogy 153
Setting convolution parameters 153
Convolutional Neural Networks (CNNs) 155
Creating convolution layers 156
Creating pooling layers 158
Putting Theory into Practice 160
Processing CIFAR images 160
Classifying CIFAR images in code 162
Performing Image Operations 166
Converting images 166
Color processing 169
Rotating and mirroring 170
Resizing and cropping 172
Convolution 174
Putting Theory into Practice 175
Chapter 9: Analyzing Sequential Data with Recurrent Neural Networks (RNNs) 179
Recurrent Neural Networks (RNNs) 180
RNNs and recursive functions 181
Training RNNs 182
Creating RNN Cells 183
Creating a basic RNN 185
Predicting text with RNNs 188
Creating multilayered cells 190
Creating dynamic RNNs 191
Long Short-Term Memory (LSTM) Cells 192
Creating LSTMs in code 194
Predicting text with LSTMs 196
Gated Recurrent Units (GRUs) 196
Creating GRUs in code 197
Predicting text with GRUs 198
Part 3: Simplifying and Accelerating Tensorflow 199
Chapter 10: Accessing Data with Datasets and Iterators 201
Datasets 201
Creating datasets 202
Processing datasets 208
Iterators 213
One-shot iterators 213
Initializable iterators 215
Reinitializable iterators 216
Feedable iterators 217
Putting Theory into Practice 218
Bizarro Datasets 221
Loading data from CSV files 222
Loading the Iris and Boston datasets 223
Chapter 11: Using Threads, Devices, and Clusters 225
Executing with Multiple Threads 226
Configuring a new session 226
Configuring a running session 228
Configuring Devices 229
Building TensorFlow from source 229
Assigning operations to devices 235
Configuring GPU usage 237
Executing TensorFlow in a Cluster 238
Creating a ClusterSpec 239
Creating a server 240
Specifying jobs and tasks 241
Running a simple cluster 244
Chapter 12: Developing Applications with Estimators 247
Introducing Estimators 248
Training an Estimator 248
Testing an Estimator 250
Running an Estimator 250
Creating Input Functions 251
Configuring an Estimator 252
Using Feature Columns 253
Creating and Using Estimators 256
Linear regressors 257
DNN classifiers 260
Combined linear-DNN classifiers 262
Wide and deep learning 263
Analyzing census data 264
Running Estimators in a Cluster 269
Accessing Experiments 270
Creating an experiment 271
Methods of the experiment class 272
Running an experiment 273
Putting theory into practice 274
Chapter 13: Running Applications on the Google Cloud Platform (GCP) 277
Overview 278
Working with GCP projects 278
Creating a new project 279
Billing 279
Accessing the machine learning engine 280
The Cloud Software Development Kit (SDK) 280
The gcloud Utility 281
Google Cloud Storage 283
Buckets 283
Objects and virtual hierarchy 285
The gsutil utility 286
Preparing for Deployment 290
Receiving arguments 290
Packaging TensorFlow code 291
Executing Applications with the Cloud SDK 293
Local execution 294
Deploying to the cloud 295
Configuring a Cluster in the Cloud 299
Setting the training input 300
Obtaining the training output 303
Setting the prediction input 304
Obtaining the prediction output 305
Part 4: The Part of Tens 307
Chapter 14: The Ten Most Important Classes 309
Tensor 309
Operation 310
Graph 310
Session 311
Variable 311
Optimizer 312
Estimator 312
Dataset 312
Iterator 313
Saver 313
Chapter 15: Ten Recommendations for Training Neural Networks 315
Select a Representative Dataset 315
Standardize Your Data 316
Use Proper Weight Initialization 316
Start with a Small Number of Layers 316
Add Dropout Layers 317
Train with Small, Random Batches 317
Normalize Batch Data 317
Try Different Optimization Algorithms 318
Set the Right Learning Rate 318
Check Weights and Gradients 318
Index 319
