Preface xvii
List of contributors xix
1 Introduction 1
Clair L. Alston, Margaret Donald, Kerrie L. Mengersen and Anthony N. Pettitt
1.1 Introduction 1
1.2 Overview 1
1.3 Further reading 8
1.3.1 Bayesian theory and methodology 8
1.3.2 Bayesian methodology 10
1.3.3 Bayesian computation 10
1.3.4 Bayesian software 11
1.3.5 Applications 13
References 13
2 Introduction to MCMC 17
Anthony N. Pettitt and Candice M. Hincksman
2.1 Introduction 17
2.2 Gibbs sampling 18
2.2.1 Example: Bivariate normal 18
2.2.2 Example: Change-point model 19
2.3 Metropolis–Hastings algorithms 19
2.3.1 Example: Component-wise MH or MH within Gibbs 20
2.3.2 Extensions to basic MCMC 21
2.3.3 Adaptive MCMC 22
2.3.4 Doubly intractable problems 22
2.4 Approximate Bayesian computation 24
2.5 Reversible jump MCMC 25
2.6 MCMC for some further applications 26
References 27
3 Priors: Silent or active partners of Bayesian inference? 30
Samantha Low Choy
3.1 Priors in the very beginning 30
3.1.1 Priors as a basis for learning 32
3.1.2 Priors and philosophy 32
3.1.3 Prior chronology 33
3.1.4 Pooling prior information 34
3.2 Methodology I: Priors defined by mathematical criteria 35
3.2.1 Conjugate priors 35
3.2.2 Impropriety and hierarchical priors 37
3.2.3 Zellner’s g-prior for regression models 37
3.2.4 Objective priors 38
3.3 Methodology II: Modelling informative priors 40
3.3.1 Informative modelling approaches 40
3.3.2 Elicitation of distributions 42
3.4 Case studies 44
3.4.1 Normal likelihood: Time to submit research dissertations 44
3.4.2 Binomial likelihood: Surveillance for exotic plant pests 47
3.4.3 Mixture model likelihood: Bioregionalization 50
3.4.4 Logistic regression likelihood: Mapping species distribution via habitat models 53
3.5 Discussion 57
3.5.1 Limitations 57
3.5.2 Finding out about the problem 58
3.5.3 Prior formulation 59
3.5.4 Communication 60
3.5.5 Conclusion 61
Acknowledgements 61
References 61
4 Bayesian analysis of the normal linear regression model 66
Christopher M. Strickland and Clair L. Alston
4.1 Introduction 66
4.2 Case studies 67
4.2.1 Case study 1: Boston housing data set 67
4.2.2 Case study 2: Production of cars and station wagons 67
4.3 Matrix notation and the likelihood 67
4.4 Posterior inference 68
4.4.1 Natural conjugate prior 69
4.4.2 Alternative prior specifications 73
4.4.3 Generalizations of the normal linear model 74
4.4.4 Variable selection 78
4.5 Analysis 81
4.5.1 Case study 1: Boston housing data set 81
4.5.2 Case study 2: Car production data set 85
References 88
5 Adapting ICU mortality models for local data: A Bayesian approach 90
Petra L. Graham, Kerrie L. Mengersen and David A. Cook
5.1 Introduction 90
5.2 Case study: Updating a known risk-adjustment model for local use 91
5.3 Models and methods 92
5.4 Data analysis and results 96
5.4.1 Updating using the training data 96
5.4.2 Updating the model yearly 98
5.5 Discussion 100
References 101
6 A Bayesian regression model with variable selection for genome-wide association studies 103
Carla Chen, Kerrie L. Mengersen, Katja Ickstadt and Jonathan M. Keith
6.1 Introduction 103
6.2 Case study: Case–control of Type 1 diabetes 104
6.3 Case study: GENICA 105
6.4 Models and methods 105
6.4.1 Main effect models 105
6.4.2 Main effects and interactions 108
6.5 Data analysis and results 109
6.5.1 WTCCC TID 109
6.5.2 GENICA 110
6.6 Discussion 112
Acknowledgements 115
References 115
6.A Appendix: SNP IDs 117
7 Bayesian meta-analysis 118
Jegar O. Pitchforth and Kerrie L. Mengersen
7.1 Introduction 118
7.2 Case study 1: Association between red meat consumption and breast cancer 119
7.2.1 Background 119
7.2.2 Meta-analysis models 121
7.2.3 Computation 125
7.2.4 Results 125
7.2.5 Discussion 129
7.3 Case study 2: Trends in fish growth rate and size 130
7.3.1 Background 130
7.3.2 Meta-analysis models 131
7.3.3 Computation 134
7.3.4 Results 134
7.3.5 Discussion 135
Acknowledgements 137
References 138
8 Bayesian mixed effects models 141
Clair L. Alston, Christopher M. Strickland, Kerrie L. Mengersen and Graham E. Gardner
8.1 Introduction 141
8.2 Case studies 142
8.2.1 Case study 1: Hot carcase weight of sheep carcases 142
8.2.2 Case study 2: Growth of primary school girls 142
8.3 Models and methods 146
8.3.1 Model for Case study 1 147
8.3.2 Model for Case study 2 148
8.3.3 MCMC estimation 149
8.4 Data analysis and results 150
8.5 Discussion 158
References 158
9 Ordering of hierarchies in hierarchical models: Bone mineral density estimation 159
Cathal D. Walsh and Kerrie L. Mengersen
9.1 Introduction 159
9.2 Case study 160
9.2.1 Measurement of bone mineral density 160
9.3 Models 161
9.3.1 Hierarchical model 162
9.3.2 Model H1 163
9.3.3 Model H2 163
9.4 Data analysis and results 164
9.4.1 Model H1 164
9.4.2 Model H2 165
9.4.3 Implication of ordering 166
9.4.4 Simulation study 166
9.4.5 Study design 166
9.4.6 Simulation study results 167
9.5 Discussion 168
References 168
9.A Appendix: Likelihoods 170
10 Bayesian Weibull survival model for gene expression data 171
Sri Astuti Thamrin, James M. McGree and Kerrie L. Mengersen
10.1 Introduction 171
10.2 Survival analysis 172
10.3 Bayesian inference for the Weibull survival model 174
10.3.1 Weibull model without covariates 174
10.3.2 Weibull model with covariates 175
10.3.3 Model evaluation and comparison 176
10.4 Case study 178
10.4.1 Weibull model without covariates 178
10.4.2 Weibull survival model with covariates 180
10.4.3 Model evaluation and comparison 182
10.5 Discussion 182
References 183
11 Bayesian change point detection in monitoring clinical outcomes 186
Hassan Assareh, Ian Smith and Kerrie L. Mengersen
11.1 Introduction 186
11.2 Case study: Monitoring intensive care unit outcomes 187
11.3 Risk-adjusted control charts 187
11.4 Change point model 188
11.5 Evaluation 189
11.6 Performance analysis 190
11.7 Comparison of Bayesian estimator with other methods 194
11.8 Conclusion 194
References 195
12 Bayesian splines 197
Samuel Clifford and Samantha Low Choy
12.1 Introduction 197
12.2 Models and methods 197
12.2.1 Splines and linear models 197
12.2.2 Link functions 198
12.2.3 Bayesian splines 198
12.2.4 Markov chain Monte Carlo 204
12.2.5 Model choice 206
12.2.6 Posterior diagnostics 207
12.3 Case studies 207
12.3.1 Data 207
12.3.2 Analysis 208
12.4 Conclusion 216
12.4.1 Discussion 216
12.4.2 Extensions 217
12.4.3 Summary 218
References 218
13 Disease mapping using Bayesian hierarchical models 221
Arul Earnest, Susanna M. Cramb and Nicole M. White
13.1 Introduction 221
13.2 Case studies 224
13.2.1 Case study 1: Spatio-temporal model examining the incidence of birth defects 224
13.2.2 Case study 2: Relative survival model examining survival from breast cancer 225
13.3 Models and methods 225
13.3.1 Case study 1 225
13.3.2 Case study 2 229
13.4 Data analysis and results 230
13.4.1 Case study 1 230
13.4.2 Case study 2 231
13.5 Discussion 234
References 237
14 Moisture, crops and salination: An analysis of a three-dimensional agricultural data set 240
Margaret Donald, Clair L. Alston, Rick Young and Kerrie L. Mengersen
14.1 Introduction 240
14.2 Case study 241
14.2.1 Data 242
14.2.2 Aim of the analysis 242
14.3 Review 243
14.3.1 General methodology 243
14.3.2 Computations 243
14.4 Case study modelling 243
14.4.1 Modelling framework 243
14.5 Model implementation: Coding considerations 246
14.5.1 Neighbourhood matrices and CAR models 246
14.5.2 Design matrices vs indexing 246
14.6 Case study results 247
14.7 Conclusions 249
References 250
15 A Bayesian approach to multivariate state space modelling: A study of a Fama–French asset-pricing model with time-varying regressors 252
Christopher M. Strickland and Philip Gharghori
15.1 Introduction 252
15.2 Case study: Asset pricing in financial markets 253
15.2.1 Data 254
15.3 Time-varying Fama–French model 254
15.3.1 Specific models under consideration 255
15.4 Bayesian estimation 256
15.4.1 Gibbs sampler 256
15.4.2 Sampling Σε 257
15.4.3 Sampling β 1:n 257
15.4.4 Sampling Σ α 259
15.4.5 Likelihood calculation 260
15.5 Analysis 261
15.5.1 Prior elicitation 261
15.5.2 Estimation output 261
15.6 Conclusion 264
References 265
16 Bayesian mixture models: When the thing you need to know is the thing you cannot measure 267
Clair L. Alston, Kerrie L. Mengersen and Graham E. Gardner
16.1 Introduction 267
16.2 Case study: CT scan images of sheep 268
16.3 Models and methods 270
16.3.1 Bayesian mixture models 270
16.3.2 Parameter estimation using the Gibbs sampler 273
16.3.3 Extending the model to incorporate spatial information 274
16.4 Data analysis and results 276
16.4.1 Normal Bayesian mixture model 276
16.4.2 Spatial mixture model 278
16.4.3 Carcase volume calculation 281
16.5 Discussion 284
References 284
17 Latent class models in medicine 287
Margaret Rolfe, Nicole M. White and Carla Chen
17.1 Introduction 287
17.2 Case studies 288
17.2.1 Case study 1: Parkinson’s disease 288
17.2.2 Case study 2: Cognition in breast cancer 288
17.3 Models and methods 289
17.3.1 Finite mixture models 290
17.3.2 Trajectory mixture models 292
17.3.3 Goodness of fit 296
17.3.4 Label switching 297
17.3.5 Model computation 298
17.4 Data analysis and results 300
17.4.1 Case study 1: Phenotype identification in PD 300
17.4.2 Case study 2: Trajectory groups for verbal memory 302
17.5 Discussion 306
References 307
18 Hidden Markov models for complex stochastic processes: A case study in electrophysiology 310
Nicole M. White, Helen Johnson, Peter Silburn, Judith Rousseau and Kerrie L. Mengersen
18.1 Introduction 310
18.2 Case study: Spike identification and sorting of extracellular recordings 311
18.3 Models and methods 312
18.3.1 What is an HMM? 312
18.3.2 Modelling a single AP: Application of a simple HMM 313
18.3.3 Multiple neurons: An application of a factorial HMM 315
18.3.4 Model estimation and inference 317
18.4 Data analysis and results 320
18.4.1 Simulation study 320
18.4.2 Case study: Extracellular recordings collected during deep brain stimulation 323
18.5 Discussion 326
References 327
19 Bayesian classification and regression trees 330
Rebecca A. O’Leary, Samantha Low Choy, Wenbiao Hu and Kerrie L. Mengersen
19.1 Introduction 330
19.2 Case studies 332
19.2.1 Case study 1: Kyphosis 332
19.2.2 Case study 2: Cryptosporidium 332
19.3 Models and methods 334
19.3.1 CARTs 334
19.3.2 Bayesian CARTs 335
19.4 Computation 337
19.4.1 Building the BCART model – stochastic search 337
19.4.2 Model diagnostics and identifying good trees 339
19.5 Case studies – results 341
19.5.1 Case study 1: Kyphosis 341
19.5.2 Case study 2: Cryptosporidium 343
19.6 Discussion 345
References 346
20 Tangled webs: Using Bayesian networks in the fight against infection 348
Mary Waterhouse and Sandra Johnson
20.1 Introduction to Bayesian network modelling 348
20.1.1 Building a BN 349
20.2 Introduction to case study 351
20.3 Model 352
20.4 Methods 354
20.5 Results 355
20.6 Discussion 357
References 359
21 Implementing adaptive dose finding studies using sequential Monte Carlo 361
James M. McGree, Christopher C. Drovandi and Anthony N. Pettitt
21.1 Introduction 361
21.2 Model and priors 363
21.3 SMC for dose finding studies 364
21.3.1 Importance sampling 364
21.3.2 SMC 365
21.3.3 Dose selection procedure 367
21.4 Example 369
21.5 Discussion 371
References 372
21.A Appendix: Extra example 373
22 Likelihood-free inference for transmission rates of nosocomial pathogens 374
Christopher C. Drovandi and Anthony N. Pettitt
22.1 Introduction 374
22.2 Case study: Estimating transmission rates of nosocomial pathogens 375
22.2.1 Background 375
22.2.2 Data 376
22.2.3 Objective 376
22.3 Models and methods 376
22.3.1 Models 376
22.3.2 Computing the likelihood 379
22.3.3 Model simulation 380
22.3.4 ABC 381
22.3.5 ABC algorithms 382
22.4 Data analysis and results 384
22.5 Discussion 385
References 386
23 Variational Bayesian inference for mixture models 388
Clare A. McGrory
23.1 Introduction 388
23.2 Case study: Computed tomography (CT) scanning of a loin portion of a pork carcase 390
23.3 Models and methods 392
23.4 Data analysis and results 397
23.5 Discussion 399
References 399
23.A Appendix: Form of the variational posterior for a mixture of multivariate normal densities 401
24 Issues in designing hybrid algorithms 403
Jeong E. Lee, Kerrie L. Mengersen and Christian P. Robert
24.1 Introduction 403
24.2 Algorithms and hybrid approaches 406
24.2.1 Particle system in the MCMC context 407
24.2.2 MALA 407
24.2.3 DRA 408
24.2.4 PS 409
24.2.5 Population Monte Carlo (PMC) algorithm 410
24.3 Illustration of hybrid algorithms 412
24.3.1 Simulated data set 412
24.3.2 Application: Aerosol particle size 415
24.4 Discussion 417
References 418
25 A Python package for Bayesian estimation using Markov chain Monte Carlo 421
Christopher M. Strickland, Robert J. Denham, Clair L. Alston and Kerrie L. Mengersen
25.1 Introduction 421
25.2 Bayesian analysis 423
25.2.1 MCMC methods and implementation 424
25.2.2 Normal linear Bayesian regression model 433
25.3 Empirical illustrations 437
25.3.1 Example 1: Linear regression model – variable selection and estimation 438
25.3.2 Example 2: Loglinear model 441
25.3.3 Example 3: First-order autoregressive regression 446
25.4 Using PyMCMC efficiently 451
25.4.1 Compiling code in Windows 455
25.5 PyMCMC interacting with R 457
25.6 Conclusions 458
25.7 Obtaining PyMCMC 459
References 459
Index 461
