Introduction 1
About This Book 1
Foolish Assumptions 2
Icons Used in This Book 3
Beyond the Book 4
Where to Go from Here 5
Part 1: Discovering Deep Learning 7
Chapter 1: Introducing Deep Learning 9
Defining What Deep Learning Means 10
Starting from Artificial Intelligence 10
Considering the role of AI 12
Focusing on machine learning 15
Moving from machine learning to deep learning 16
Using Deep Learning in the Real World 18
Understanding the concept of learning 18
Performing deep learning tasks 19
Employing deep learning in applications 19
Considering the Deep Learning Programming Environment 19
Overcoming Deep Learning Hype 22
Discovering the start-up ecosystem 22
Knowing when not to use deep learning 22
Chapter 2: Introducing the Machine Learning Principles 25
Defining Machine Learning 26
Understanding how machine learning works 26
Understanding that it's pure math 27
Learning by different strategies 28
Training, validating, and testing data 30
Looking for generalization 31
Getting to know the limits of bias 32
Keeping model complexity in mind 33
Considering the Many Different Roads to Learning 33
Understanding there is no free lunch 34
Discovering the five main approaches 34
Delving into some different approaches 36
Awaiting the next breakthrough 40
Pondering the True Uses of Machine Learning 40
Understanding machine learning benefits 41
Discovering machine learning limits 43
Chapter 3: Getting and Using Python 45
Working with Python in this Book 46
Obtaining Your Copy of Anaconda 46
Getting Continuum Analytics Anaconda 47
Installing Anaconda on Linux 47
Installing Anaconda on MacOS 48
Installing Anaconda on Windows 49
Downloading the Datasets and Example Code 54
Using Jupyter Notebook 54
Defining the code repository 56
Getting and using datasets 61
Creating the Application 62
Understanding cells 62
Adding documentation cells 63
Using other cell types 64
Understanding the Use of Indentation 65
Adding Comments 66
Understanding comments 67
Using comments to leave yourself reminders 68
Using comments to keep code from executing 69
Getting Help with the Python Language 69
Working in the Cloud 70
Using the Kaggle datasets and kernels 70
Using the Google Colaboratory 70
Chapter 4: Leveraging a Deep Learning Framework 73
Presenting Frameworks 74
Defining the differences 74
Explaining the popularity of frameworks 75
Defining the deep learning framework 77
Choosing a particular framework 78
Working with Low-End Frameworks 79
Caffe2 79
Chainer 80
PyTorch 80
MXNet 81
Microsoft Cognitive Toolkit/CNTK 82
Understanding TensorFlow 82
Grasping why TensorFlow is so good 82
Making TensorFlow easier by using TFLearn 84
Using Keras as the best simplifier 85
Getting your copy of TensorFlow and Keras 86
Fixing the C++ build tools error in Windows 88
Accessing your new environment in Notebook 89
Part 2: Considering Deep Learning Basics 91
Chapter 5: Reviewing Matrix Math and Optimization 93
Revealing the Math You Really Need 94
Working with data 94
Creating and operating with a matrix 95
Understanding Scalar, Vector, and Matrix Operations 96
Creating a matrix 97
Performing matrix multiplication 99
Executing advanced matrix operations 100
Extending analysis to tensors 102
Using vectorization effectively 104
Interpreting Learning as Optimization 105
Exploring cost functions 105
Descending the error curve 106
Learning the right direction 107
Updating 109
Chapter 6: Laying Linear Regression Foundations 111
Combining Variables 112
Working through simple linear regression 112
Advancing to multiple linear regression 113
Including gradient descent 115
Seeing linear regression in action 116
Mixing Variable Types 117
Modeling the responses 117
Modeling the features 118
Dealing with complex relations 119
Switching to Probabilities 121
Specifying a binary response 121
Transforming numeric estimates into probabilities 122
Guessing the Right Features 124
Defining the outcome of incompatible features 124
Solving overfitting using selection and regularization 125
Learning One Example at a Time 127
Using gradient descent 127
Understanding how SGD is different 127
Chapter 7: Introducing Neural Networks 131
Discovering the Incredible Perceptron 132
Understanding perceptron functionality 132
Touching the nonseparability limit 134
Hitting Complexity with Neural Networks 136
Considering the neuron 136
Pushing data with feed-forward 138
Going even deeper into the rabbit hole 140
Using backpropagation to adjust learning 143
Struggling with Overfitting 146
Understanding the problem 146
Opening the black box 146
Chapter 8: Building a Basic Neural Network 149
Understanding Neural Networks 150
Defining the basic architecture 151
Documenting the essential modules 153
Solving a simple problem 155
Looking Under the Hood of Neural Networks 158
Choosing the right activation function 158
Relying on a smart optimizer 160
Setting a working learning rate 161
Chapter 9: Moving to Deep Learning 163
Seeing Data Everywhere 164
Considering the effects of structure 164
Understanding Moore's implications 165
Considering what Moore's Law changes 166
Discovering the Benefits of Additional Data 167
Defining the ramifications of data 168
Considering data timeliness and quality 168
Improving Processing Speed 169
Leveraging powerful hardware 170
Making other investments 170
Explaining Deep Learning Differences from Other Forms of AI 171
Adding more layers 172
Changing the activations 174
Adding regularization by dropout 175
Finding Even Smarter Solutions 176
Using online learning 176
Transferring learning 177
Learning end to end 177
Chapter 10: Explaining Convolutional Neural Networks 179
Beginning the CNN Tour with Character Recognition 180
Understanding image basics 180
Explaining How Convolutions Work 183
Understanding convolutions 183
Simplifying the use of pooling 187
Describing the LeNet architecture 188
Detecting Edges and Shapes from Images 193
Visualizing convolutions 194
Unveiling successful architectures 196
Discussing transfer learning 197
Chapter 11: Introducing Recurrent Neural Networks 201
Introducing Recurrent Networks 202
Modeling sequences using memory 202
Recognizing and translating speech 204
Placing the correct caption on pictures 206
Explaining Long Short-Term Memory 207
Defining memory differences 208
Walking through the LSTM architecture 209
Discovering interesting variants 211
Getting the necessary attention 212
Part 3: Interacting with Deep Learning 215
Chapter 12: Performing Image Classification 217
Using Image Classification Challenges 218
Delving into ImageNet and MS COCO 219
Learning the magic of data augmentation 221
Distinguishing Traffic Signs 223
Preparing image data 224
Running a classification task 228
Chapter 13: Learning Advanced CNNs 233
Distinguishing Classification Tasks 234
Performing localization 235
Classifying multiple objects 235
Annotating multiple objects in images 237
Segmenting images 237
Perceiving Objects in Their Surroundings 239
Discovering how RetinaNet works 239
Using the Keras-RetinaNet code 241
Overcoming Adversarial Attacks on Deep Learning Applications 245
Tricking pixels 246
Hacking with stickers and other artifacts 248
Chapter 14: Working on Language Processing 251
Processing Language 252
Defining understanding as tokenization 253
Putting all the documents into a bag 254
Memorizing Sequences that Matter 257
Understanding semantics by word embeddings 257
Using AI for Sentiment Analysis 261
Chapter 15: Generating Music and Visual Art 269
Learning to Imitate Art and Life 270
Transferring an artistic style 271
Reducing the problem to statistics 272
Understanding that deep learning doesn't create 274
Mimicking an Artist 274
Defining a new piece based on a single artist 274
Combining styles to create new art 276
Visualizing how neural networks dream 276
Using a network to compose music 277
Chapter 16: Building Generative Adversarial Networks 279
Making Networks Compete 280
Finding the key in the competition 280
Achieving more realistic results 282
Considering a Growing Field 289
Inventing realistic pictures of celebrities 289
Enhancing details and image translation 290
Chapter 17: Playing with Deep Reinforcement Learning 293
Playing a Game with Neural Networks 294
Introducing reinforcement learning 294
Simulating game environments 296
Presenting Q-learning 299
Explaining Alpha-Go 302
Determining if you're going to win 303
Applying self-learning at scale 305
Part 4: The Part of Tens 307
Chapter 18: Ten Applications that Require Deep Learning 309
Restoring Color to Black-and-White Videos and Pictures 310
Approximating Person Poses in Real Time 310
Performing Real-Time Behavior Analysis 311
Translating Languages 312
Estimating Solar Savings Potential 312
Beating People at Computer Games 313
Generating Voices 314
Predicting Demographics 314
Creating Art from Real-World Pictures 315
Forecasting Natural Catastrophes 316
Chapter 19: Ten Must-Have Deep Learning Tools 317
Compiling Math Expressions Using Theano 317
Augmenting TensorFlow Using Keras 318
Dynamically Computing Graphs with Chainer 319
Creating a MATLAB-Like Environment with Torch 319
Performing Tasks Dynamically with PyTorch 320
Accelerating Deep Learning Research Using CUDA 321
Supporting Business Needs with Deeplearning4j 323
Mining Data Using Neural Designer 323
Training Algorithms Using Microsoft Cognitive Toolkit (CNTK) 324
Exploiting Full GPU Capability Using MXNet 325
Chapter 20: Ten Types of Occupations that Use Deep Learning 327
Managing People 327
Improving Medicine 328
Developing New Devices 329
Providing Customer Support 329
Seeing Data in New Ways 330
Performing Analysis Faster 331
Creating a Better Work Environment 331
Researching Obscure or Detailed Information 333
Designing Buildings 333
Enhancing Safety 334
Index 335
