Preface xv
Acknowledgments xxi
About the Companion Website xxiii
1 Assessment and Uncertainty: Examples and Introductory Concepts 1
1.1 Chicken Little, Amateur Meteorologist 2
1.2 Uncertainty Ascribes Meaning to Values 3
1.3 Significant Figures 3
1.4 Types of Uncertainty 7
1.5 Example: Finding Saturn¢s Moons 9
1.6 Comparing Two Numbers: Are They Measuring the Same Value? 11
1.6.1 Distributions of Number Sets 12
1.6.2 The Gaussian Distribution 13
1.6.3 Testing a Specific Value within a Data Set: The z Test 14
1.6.4 Comparing Two Values Revisited 18
1.7 Use and Misuse of Statistics 19
1.8 Example: Solar Wind Density and Space Weather 20
1.9 Uncertainty and the Scientific Method 22
1.10 Further Reading 24
1.11 Exercises in the Geosciences 26
2 Plotting Data: Visualizing Sets of Numbers 27
2.1 Plotting One- Dimensional Data 27
2.1.1 What Makes a Good Plot? 29
2.1.2 Exploratory Versus Explanatory Plot Styles 31
2.2 Example: Earth¢s Magnetic Field Strength 33
2.3 Probability Distributions¢" The Histogram 35
2.4 Plotting Two Data Sets Against Each Other 39
2.4.1 Overlaid Histograms 39
2.4.2 The Scatterplot 40
2.4.3 The Box Plot 42
2.4.4 The Box- and- Whisker Scatterplot 43
2.4.5 The Running Average Plot 44
2.5 Example: Temperature and Carbon Dioxide 48
2.6 Scientific Visualization: A Sampling from the Literature 50
2.6.1 A Very Brief History of Visualization 51
2.6.2 Good Modern- Day Example Visualizations 53
2.7 Visualization Best Practices 58
2.7.1 Levels of Abstraction 58
2.7.2 A Process for a Good Graphic 61
2.7.3 Types of Colorblindness 63
2.7.4 Color Scales 63
2.8 Further Reading 65
2.9 Exercises in the Geosciences 67
3 Uncertainty Analysis: Techniques for Propagating Uncertainty 69
3.1 Propagating Uncertainty 69
3.1.1 Calculating Uncertainty with One Independent Variable 69
3.1.2 Calculating Uncertainty with Two Independent Variables 70
3.1.3 Calculating Uncertainty with Many Independent Variables 72
3.2 Example: Atmospheric Density 72
3.2.1 The Hydrostatic Equilibrium Approximation 72
3.2.2 One Independent Variable 73
3.2.3 Two Independent Variables 74
3.2.4 Many Independent Variables 74
3.3 Fractional and Percentage Uncertainties 75
3.4 Special Cases of Uncertainty Propagation 77
3.4.1 Addition and Subtraction 77
3.4.2 Multiplication and Division 78
3.4.2.1 Multiplication of Two Parameters 78
3.4.2.2 Uncertainty of Air Pressure 79
3.4.2.3 Division with Correlated Variables 80
3.4.2.4 Multiplication and Division with Independent Variables 81
3.4.3 Power Laws 82
3.4.4 Exponentials and Logarithms 82
3.4.4.1 Exponential Functions 83
3.4.4.2 Logarithmic Functions 84
3.4.5 Trigonometric Functions 84
3.5 Stepwise Uncertainty Propagation 85
3.6 Example: Planetary Equilibrium Temperature 87
3.7 Multistep Processing 90
3.8 Final Advice on Uncertainty Propagation 91
3.9 Further Reading 93
3.10 Exercises in the Geosciences 93
4 Centroids and Spreads: Analyzing a Set of Numbers 95
4.1 Quantitatively Describing a Data Set: The Centroid 95
4.1.1 Three Versions of Mean 96
4.1.2 More Centroids: Median and Mode 98
4.1.3 Histograms and the Arithmetic Mean 99
4.2 Quantitatively Describing a Data Set: Spread 100
4.2.1 Measures of Spread: Standard Deviation and Mean Absolute Difference 100
4.2.2 Another Measure of Spread: Quantiles 102
4.2.3 Spread Via Full Width at Half Maximum 106
4.2.4 Spread as an L- p Norm 107
4.2.5 Sample Versus Population 108
4.3 Random and Systematic Error of a Data Set 109
4.4 Which Centroid and Spread to Use and Other Tidbits of Advice 111
4.5 Standard Deviation of the Mean 112
4.6 Counting Statistics 113
4.7 Example: Galactic Cosmic Rays 116
4.8 Further Reading 119
4.9 Exercises in the Geosciences 120
5 Assessing Normality: Tests for Assessing the Gaussian Nature of a Distribution 123
5.1 Histogram Check 124
5.2 Comparing Centroid and Spread Measures 126
5.3 Skew 128
5.4 Kurtosis 130
5.5 The Chi- Squared Test 132
5.6 The Kolmogorov¢"Smirnov Test 137
5.7 Example: pH in a Lake 139
5.8 Asymmetric Uncertainties 142
5.9 Outliers¢" Tests for a Single Data Value 144
5.10 Combining Centroid and Spread: The Weighted Average 146
5.11 Example: pH in a Lake Redux 148
5.12 Further Reading 149
5.13 Exercises in the Geosciences 150
6 Correlating Two Data Sets: Analyzing Two Sets of Numbers Together 153
6.1 Comparing Two Number Sets 153
6.1.1 Chi- Squared and Kolmogorov¢"Smirnov Tests 154
6.1.2 The Student¢s t Test 155
6.1.3 The Welch¢s t Test 156
6.2 Linear Correlation 157
6.2.1 Covariance of Two Data Sets 158
6.2.2 Pearson Linear Correlation Coefficient 161
6.2.3 Spearman Rank- Order Correlation 163
6.2.4 Correlation with Logarithms 167
6.3 Example: Atmospheric Ozone and Temperature 168
6.4 Uncertainty of R 172
6.4.1 The Jackknife Method 172
6.4.2 The Bootstrap Method 173
6.4.3 Uncertainty of R for the Ozone- Temperature Example 175
6.5 Correlation and Causation 177
6.6 Further Reading 178
6.7 Exercises in the Geosciences 179
7 Curve Fitting: Fitting a Line between Two Sets of Numbers 181
7.1 Linear Regression 181
7.1.1 Obtaining A and B 181
7.1.2 Uncertainties on A and B 185
7.1.3 The Zero- Intercept Special Case 186
7.1.4 Weighted Linear Fitting 187
7.2 Testing a Linear Fit 188
7.3 Example: Human- Induced Seismicity 191
7.4 Nonlinear Fitting 194
7.4.1 Polynomial Fitting 194
7.4.2 Generalized ¢Linear Coefficient¢ Fitting 196
7.4.3 Exponential Fitting: Linearizing the Dependence on Coefficients 197
7.4.4 Piecewise Linear Fitting 198
7.4.5 Advice about Curve Fitting 199
7.5 Example: The Ozone Hole 200
7.6 Iterative Curve Fitting 203
7.6.1 One- Dimensional Iterative Curve Fitting 203
7.6.2 Multidimensional Iterative Curve Fitting 205
7.6.3 Gradient Descent Curve Fitting 208
7.7 Final Thoughts on Curve Fitting 209
7.8 Further Reading 210
7.9 Exercises in the Geosciences 210
8 Data- Model Comparison Basics: Philosophies of Calculating and Categorizing Metrics 213
8.1 Example Model: River Flow Rate 213
8.2 What Is a Model? 214
8.3 Visualizing Observed and Modeled Values Together 217
8.3.1 Scatterplots of Data and Model Values 217
8.3.2 The 2D Histogram Plot 219
8.3.3 Overlaid Histogram Plots 221
8.3.4 Cumulative Probability Distribution Plots 222
8.3.5 Quantile¢"Quantile Plots 224
8.4 Example: Total Solar Irradiance 226
8.5 A Diverse Zoo of Metrics 229
8.5.1 The Primary Categories of Metrics 230
8.5.2 Skill 231
8.5.3 Metrics Categories Based on Subsetting 234
8.6 The Concept of Model ¢Goodness of Fit¢ 235
8.7 Application Usability Levels 236
8.8 Designing a Meaningful Data- Model Comparison 237
8.9 Further Reading 239
8.10 Exercises in the Geosciences 240
9 Fit Performance Metrics: Data- Model Comparisons Based on Exact Observed and Modeled Values 243
9.1 What Is Fit Performance? 244
9.2 Running Example: Dst and the O¢Brien Model 245
9.3 Accuracy 250
9.3.1 The Big Three of Accuracy: MSE, RMSE, and MAE 251
9.3.2 Neglecting Degrees of Freedom 253
9.3.3 Normalizing the Accuracy Measure 256
9.3.4 Percentage Accuracy Metrics 257
9.3.5 Choosing the Right Accuracy Metric 261
9.4 Bias 262
9.4.1 Mean Error 262
9.4.2 Percentage Bias 265
9.5 Precision 266
9.5.1 Modeling Yield 266
9.5.2 Definitions of Precision Using Standard Deviation 268
9.6 Association 268
9.6.1 Correlation Coefficient 269
9.6.2 Nonlinear Association Metrics 270
9.7 Extremes 272
9.7.1 Extremes of the Cumulative Probability Distribution 272
9.7.2 Using Skew and Kurtosis for an Extremes Assessment 276
9.8 Skill 278
9.8.1 Prediction Efficiency 278
9.8.2 Other Options for Fit Performance Skill 279
9.9 Discrimination 281
9.10 Reliability 283
9.11 Summarizing the Running Example 286
9.12 Summary of Fit Performance Metrics 287
9.13 Further Reading 291
9.14 Exercises in the Geosciences 292
10 Event Detection Metrics: Comparing Observed and Modeled Number Sets When Only Event Status Matters 295
10.1 Defining an Event 296
10.2 Contingency Tables 299
10.3 Data- Model Comparisons with Events 301
10.4 Running Example: Will It Rain? 303
10.5 Significance of a Contingency Table 307
10.6 Accuracy 310
10.7 Bias 311
10.8 Precision 313
10.9 Association 314
10.9.1 Odds Ratio 315
10.9.2 Odds Ratio Skill Score 316
10.9.3 Matthews Correlation Coefficient 317
10.10 Extremes 317
10.11 Skill 321
10.11.1 Heidke Skill Score 321
10.11.2 Peirce and Clayton Skill Scores 323
10.11.3 Gilbert Skill Score 324
10.12 Discrimination 325
10.13 Reliability 326
10.14 Summarizing the Running Example 327
10.15 Summary of Event Detection Metrics 328
10.16 Further Reading 330
10.17 Exercises in the Geosciences 331
11 Sliding Thresholds: Event Detection Metrics with a Variable Event Identification 333
11.1 Sliding the Event Identification Thresholds 334
11.2 Sweeping the Modeled Threshold 337
11.3 Sweeping the Data Threshold 340
11.4 Sweeping Both Thresholds Simultaneously 342
11.5 Metric- Versus- Metric Curves 344
11.5.1 ROC Curves 344
11.5.2 Alt- ROC Curves 346
11.5.3 STONE Curves 347
11.6 Application of Sliding Thresholds to the Geophysical Running Examples 349
11.6.1 Event Definitions for the Running Examples 349
11.6.2 Metric- Versus- Modeled Threshold Curves for the Running Examples 352
11.6.3 Metric- Versus- Observed Threshold Curves for the Running Examples 355
11.6.4 Metric- Versus- Simultaneous Threshold Sweep Curves for the Running Examples 357
11.6.5 Metric- Versus- Metric Analysis for the Running Examples 359
11.7 The Power of Sliding Thresholds 362
11.8 Further Reading 364
11.9 Exercises in the Geosciences 365
12 Applications of Metrics and Uncertainty: Final Advice and Introductions to Advanced Topics 367
12.1 Choosing the Right Set of Metrics 367
12.1.1 Metrics for Fit Performance Assessment on Gaussian Distributions 368
12.1.2 Metrics for Fit Performance Assessment on Non- Gaussian Distributions 369
12.1.3 Metrics for Event Detection Assessment 372
12.2 Combining Metrics for Robust Data- Model Comparisons 374
12.2.1 The Accuracy¢"Bias¢"Precision Trifecta 374
12.2.2 The Accuracy¢"Association Connection 376
12.2.3 The Association¢"Extremes Linkage 377
12.2.4 Expanding Our Understanding of Skill 378
12.2.5 Using Discrimination and Reliability Together 379
12.3 Uncertainty on Metrics 380
12.4 Uncertainty on Fit Performance Metrics for the Dst Running Example 381
12.5 A Recipe for Robust Comparisons 385
12.6 Metrics and Decision- Making 387
12.6.1 Choice Combination Statistics 388
12.6.2 Example: Spacecraft- Charging Model 390
12.7 Additional Advanced Topics 392
12.7.1 Periodicity Analysis 392
12.7.2 Time- Lagged Analysis 393
12.7.3 Additional Tests 394
12.7.4 Multidimensional Data Analysis 394
12.7.5 Multidimensional Data- Model Comparisons 396
12.7.6 Uncertainty Quantification 397
12.7.7 Design of Experiments 398
12.7.8 Geographical Information System (GIS) Analysis 398
12.7.9 Machine Learning 399
12.8 Uncertainty and the Scientist 400
12.9 Further Reading 402
12.10 Exercises in Geoscience 406
Index 407
