Prologue 1
Berkeley, 1930s 1
Pattern Recognition 2
Nelson Peltz 3
Committing to One Percent 5
The Big Data Revolution 6
Introduction 7
Storytelling 7
Objective 7
Outline 8
Part I “The Revolution Starts Now: 9 Industries Transforming with Data” 8
Part II “Learning from Patterns in Big Data” 11
Part III “Leading the Revolution” 11
Storytelling (Continued) 13
Part I: the Revolution Starts Now: 9 Industries Transforming With Data 15
Chapter 1: Transforming Farms with Data 17
California, 2013 17
Brief History of Farming 18
The Data Era 19
Potato Farming 20
Precision Farming 21
Capturing Farm Data 22
Deere & Company Versus Monsanto 24
Integrated Farming Systems 25
Data Prevails 26
The Climate Corporation 26
Growsafe Systems 27
Farm of the Future 27
California, 2013 (Continued) 29
Chapter 2: Why Doctors Will Have Math Degrees 31
United States, 2014 31
The History of Medical Education 32
Scientific Method 32
Rise of Specialists 33
We Have a Problem 34
Ben Goldacre 35
Vinod Khosla 35
The Data Era 36
Collecting Data 36
Telemedicine 38
Innovating with Data 40
Implications of a Data-Driven Medical World 42
The Future of Medical School 42
A Typical Medical School 42
A Medical School for the Data Era 43
United States, 2030 44
Chapter 3: Revolutionizing Insurance: Why Actuaries Will Become Data Scientists 45
Middle of Somewhere, 2012 45
Short History of Property & Casualty Insurance and Underwriting 46
Actuarial Science In Insurance 47
Pensions, Insurance, Leases 49
Compound Interest 50
Probability 50
Mortality Data 50
Modern-Day Insurance 51
Eight Weeks to Eight Days 51
Online Policies 52
The Data Era 52
Dynamic Risk Management 52
Catastrophe Risk 54
Open Access Modeling 55
Opportunities 56
Middle of Somewhere, 2012 (Continued) 58
Chapter 4: Personalizing Retail and Fashion 59
Karolina 59
A Brief History of Retail 60
Retail Eras 60
Aristide Boucicaut 61
The Shift 62
The Data Era 63
Stitch Fix 63
Keaton Row 65
Zara 66
Karolina (Continued) 67
Chapter 5: Transforming Customer Relationships with Data 69
Buying a House 69
Brief History of Customer Service 70
Customer Service Over Time 70
Boeing 72
Financial Services 74
The Data Era 75
An Automobile Manufacturer 76
Zendesk 76
Buying a House (Continued) 77
Chapter 6: Intelligent Machines 79
Denmark 79
Intelligent Machines 80
Machine Data 81
The Data Era 82
General Electric 82
Drones 84
Tesla 86
Networks of Data 87
Denmark (Continued) 88
Chapter 7: Government and Society 89
Egypt, 2011 89
Social Media 90
Intelligence 90
Snowden Effect 91
Privacy Risk Versus Reward 91
Observation or Surveillance 93
Development Targets 93
Open Data 95
Hackathons 95
Open Access 95
Ensuring Personal Protection 96
Private Clouds 97
Sanitizing Data 97
Evidence-Based Policy 97
Public-Private Partnerships 98
Impact Bonds 101
Social Impact Bond 102
Development Impact Bonds 103
The Role of Big Data 104
Egypt, 2011 (Continued) 105
Chapter 8: Corporate Sustainability 107
City of London 107
Global Megaforces 109
Population 109
Carbon Footprint 110
Water Scarcity 110
Environmental Risk 111
BP and Exxon Mobile 111
Early Warning Systems 112
Social Media 113
Risk and Resilience 114
Measuring Sustainability 115
Long-Term Decision Making 116
Stranded Assets 117
City of London (Continued) 118
Chapter 9: Weather and Energy 119
India, 2012 119
The Weather 120
Forecasting the Weather 120
When are Weather Forecasts Wrong? 121
Chaos 122
Ensemble Forecasts 122
Communication 123
Renewable Energy 124
Solar, Hydro, and Wind Power 124
Volatile or Intermittent Supply 125
Energy Consumption 126
Smart Meters 127
Intelligent Demand-Side Management 128
India, 2012 (Continued) 129
Part II: Learning From Patterns in Big Data 131
Chapter 10: Pattern Recognition 133
Elements of Success Rhyme 133
Pattern Recognition: A Gift or Trap? 134
What Fish Teach Us About Pattern Recognition 135
Bayes’ Theorem 135
Tsukiji Market 135
Pattern Recognition 137
Rochester Institute of Technology 137
A Method for Recognizing Patterns 137
Elements of Success Rhyme (Continued) 140
Chapter 11: Why Patterns in Big Data Have Emerged 141
Meatpacking District 141
Business Models in the Data Era 142
Data as a Competitive Advantage 143
Data Improves Existing Products or Services 145
Data as the Product 145
Dun & Bradstreet 146
CoStar 148
Ihs 149
Meatpacking District (Continued) 151
Chapter 12: Patterns in Big Data 153
The Data Factor 154
Summary of Big Data Patterns 155
Redefining a Skilled Worker 155
Creating and Utilizing New Sources of Data 156
Building New Data Applications 157
Transforming and Creating New Business Processes 157
Data Collection for Competitive Advantage 158
Exposing Opinion-Based Biases 159
Real-Time Monitoring and Decision Making 159
Social Networks Leveraging and Creating Data 160
Deconstructing the Value Chain 161
New Product Offerings 161
Building for Customers Instead of Markets 162
Tradeoff Between Privacy and Insight 163
Changing the Definition of a Product 163
Inverting the Search Paradigm for Data Discovery 164
Data Security 165
New Partnerships Founded on Data 165
Shortening the Innovation Lifecycle 166
Defining New Channels to Market 166
New Economic Models 167
Forecasting and Predicting Future Events 168
Changing Incentives 168
New Partnerships (Public/Private) 169
Real-Time Monitoring and Decision Making (Early Warning Systems) 169
A Framework for Big Data Patterns 170
Part III: Leading the Revolution 171
Chapter 13: The Data Opportunity 173
What Oil Teaches Us About Data 173
Bain Study 175
Seizing the Opportunity 176
Chapter 14: Porsche 177
Rome 177
Ferdinand Porsche 178
The Birth of Porsche 178
The Porsche Sports Car 179
Porsche Today 180
Rome (Continued) 180
Chapter 15: Puma 181
Herzogenaurach 181
Advertising Wars 182
Jochen Zeitz 182
Environmental Profit and Loss 183
Herzogenaurach (Continued) 184
Chapter 16: A Methodology for Applying Big Data Patterns 185
Introduction 185
The Method 186
Step 1: Understand Data Assets 187
The Patterns 188
Step 2: Explore Data 191
Challenges 192
Questions 192
Hypotheses 193
Data 193
Models 193
Statistical Significance 194
Step 3: Design the Future 194
The Patterns 195
Step 4: Design a Data-Driven Business Model 197
The Patterns 197
Step 5: Transform Business Processes for the Data Era 199
The Patterns 199
Step 6: Design for Governance and Security 201
The Patterns 201
Step 7: Share Metrics and Incentives 202
Chapter 17: Big Data Architecture 205
Introduction 205
Architect for the Future 206
Lessons from Stuttgart 207
Big Data Reference Architectures 207
Leveraging Investments in Architecture 208
Big Data Reference Architectures 211
Business View 212
Logical View 213
Chapter 18: Business View Reference Architecture 215
Introduction 215
Men’s Trunk: A Retailer in the Data Era 216
The Business View Reference Architecture 217
Answer Fabric 218
Data Virtualization 219
Data Engines 220
Management 221
Data Governance 221
User Interface, Applications, and Business Processes 222
Summary 222
Chapter 19: Logical View Reference Architecture 223
Introduction 223
Men’s Trunk: A Retailer in the Data Era (Continued) 224
The Logical View Reference Architecture 226
Data Ingest 227
Analytics 227
Discovery 228
Landing 228
Operational Warehouse 229
Information Insight 230
Operational Data 231
Governance 231
Men’s Trunk: A Retailer in the Data Era (Continued) 232
Chapter 20: The Architecture of the Future 233
Men’s Trunk: A Retailer in the Data Era (Continued) 233
Men’s Trunk: Applying the Methodology 235
Step 1: Understand Data Assets 235
Step 2: Explore the Data 236
Step 3: Design the Future 237
Step 4: Design a Data-Driven Business Model 237
Step 5: Transform Business Processes for the Data Era 237
Step 6: Design for Governance and Security 237
Step 7: Share Metrics and Incentives 238
Men’s Trunk: The Business View Reference Architecture 239
Answer Fabric 240
Data Virtualization 241
Data Engines 241
Management 242
Data Governance 242
User Interface, Applications, and Business Processes 243
Men’s Trunk: The Logical View Reference Architecture 244
Approach 244
Men’s Trunk: A Retailer in the Data Era (Continued) 248
Epilogue 249
The Time is Now 249
Taking Action 250
Fear not Usual Competitors 251
The Future 252
Index 255
