Acknowledgments xxi
I Background 1
1 Introduction 3
1.1 Why Measure the Internet? 5
1.2 How to Read this Book 6
1.3 Resources for More Information 10
2 Internet Architecture 13
2.1 The Internetâs Architecture 13
2.1.1 The History of the Internet 14
2.1.2 The Organization of the Internet 17
2.1.3 Design Principles of the Internet 22
2.2 Details of Internet Operation 25
2.2.1 Endsystems, Links, and Routers 25
2.2.2 Autonomous Systems 26
2.2.3 Routing 26
2.3 Protocols 31
2.3.1 IP 32
2.3.2 TCP 33
2.3.3 UDP 34
2.3.4 Routing Protocols 35
2.3.5 ICMP 35
2.3.6 SNMP 36
2.3.7 IP Multicast 37
2.3.8 DNS 37
2.3.9 HTTP 39
2.3.10 P2P 40
2.4 Applications 42
3 Analytic Background 45
3.1 Linear Algebra 45
3.2 Probability 48
3.2.1 Background 49
3.2.2 Special Issues in the Internet 55
3.3 Statistics 58
3.3.1 Background 58
3.3.2 Special Issues in the Internet 61
3.4 Graphs 64
3.4.1 Background 64
3.4.2 Special Issues in the Internet 66
3.5 Metrics 70
3.6 Measurement and Modeling 73
3.6.1 Models in General 73
3.6.2 The Use of Probability Models 76
4 Practical Issues in Internet Measurement 79
4.1 Where can Measurements be Made? 80
4.1.1 Local Area Network 82
4.1.2 Inside a Backbone 82
4.1.3 Entry Points into a Network 84
4.1.4 Mirror Sites/Network Exchange Points 86
4.1.5 Wide Area Network 87
4.2 Role of Time 89
4.2.1 Background 90
4.2.2 Sources of Time Information 91
4.2.3 Synchronized Time 93
4.3 Role of Internet Directories and Databases 94
4.3.1 Internet Address and Routing Registries 95
4.3.2 Domain Name System 97
4.3.3 Measurement-related Issues in Dealing with Databases 98
4.4 Measurements Across Various Protocol Layers 99
4.4.1 Issues in Capturing Data 99
4.4.2 Changes to Infrastructure/Instrumentation 102
4.4.3 Local vs Remote vs Distributed Data Gathering 103
4.4.4 Measurement on Overlays 104
II In Depth 105
5 Infrastructure 107
5.1 Properties 107
5.1.1 Physical Device Properties 107
5.1.2 Topology Properties 111
5.1.3 Interaction of Traffic and Network 112
5.2 Challenges 115
5.2.1 Core Simplicity 115
5.2.2 Hidden Layers 116
5.2.3 Hidden Pieces 116
5.2.4 Administrative Barriers 117
5.3 Tools 117
5.3.1 Active Measurement 118
5.3.2 Passive Measurement 124
5.3.3 Fused Measurements 127
5.3.4 Bandwidth Measurement 127
5.3.5 Latency Measurement and Estimation 136
5.3.6 Geolocation 142
5.3.7 Inference 147
5.3.8 Other Tools 152
5.4 State of the Art 152
5.4.1 Equipment Properties 153
5.4.2 Topology Properties 154
5.4.3 Interaction of Traffic and Network 165
6 Traffic 171
6.1 Properties 172
6.1.1 The Basics: Packets and Bytes 172
6.1.2 Higher-level Structure 173
6.1.3 Flows 175
6.1.4 Semantically Distinct Traffic Types 176
6.2 Challenges 176
6.2.1 Practical Issues 177
6.2.2 Statistical Difficulties 179
6.3 Tools 188
6.3.1 Packet Capture 188
6.3.2 Data Management 191
6.3.3 Data Reduction 192
6.3.4 Inference 212
6.4 State of the Art 215
6.4.1 Packets and Bytes 216
6.4.2 Higher-level Structure 234
6.4.3 Flows 236
6.4.4 Control Traffic 238
6.4.5 Wireless 239
7 Applications 241
7.1 Application Mix 242
7.2 DNS 244
7.2.1 DNS Measurement Properties 245
7.2.2 DNS Measurement Challenges 248
7.2.3 DNS Measurement Tools 251
7.2.4 Use of DNS in Other Applications 256
7.2.5 State of the Art 258
7.3 Web 269
7.3.1 Web Measurement Properties 270
7.3.2 Web Measurement Challenges 273
7.3.3 Web Measurement Tools 278
7.3.4 State of the Art 286
7.4 P2P 309
7.4.1 P2P Measurement Properties 310
7.4.2 P2P Measurement Challenges 314
7.4.3 P2P Measurement Tools 317
7.4.4 State of the Art 321
7.5 Online Games 331
7.5.1 Games and Measurement Properties 332
7.5.2 Networked Games Measurement Challenges 337
7.5.3 State of the Art 340
7.6 Other Applications 346
7.6.1 Streaming Multimedia 346
III In Perspective 353
8 Anonymization 355
8.1 Definitions 356
8.2 General Motivation for Anonymizing Data 357
8.3 Obstacles and Risks in Sharing Data 358
8.4 What Should be Anonymized: Data Categorization 360
8.5 How Data is Anonymized: Process and Techniques 365
8.5.1 Anonymization Process 365
8.5.2 Anonymization Techniques 367
8.6 Anonymization Examples at Different Layers 369
8.6.1 Configuration Data 369
8.6.2 Router-level Data 370
8.6.3 Packet-level Traces 370
8.6.4 Application-level Data 373
8.7 Attacks Against Anonymized Data 374
8.8 Anonymizing Data: Metrics for Success 376
8.9 Alternatives to Anonymization 377
9 Security 379
9.1 Role of Internet Measurement in Security 380
9.2 Intranet Measurements in Aid of Security 382
9.3 Gateway Measurements in Aid of Security 384
9.4 Inter-domain Measurements Impact on Security 386
9.5 Wide-area Measurements in Aid of Security 387
9.6 Application-level Measurements of Attacks 394
10 Case Studies 395
10.1 Low-level Monitoring Tools 395
10.2 Individual Toolsets for Network Measurement 397
10.2.1 Windmill 398
10.2.2 Click 399
10.2.3 dss 400
10.2.4 Gigascope 403
10.3 Large-scale Measurement Projects 404
10.3.1 RIPE 405
10.3.2 High-energy Physics 407
10.3.3 CAIDA 410
10.3.4 PlanetLab 414
11 Conclusions and Prospects 419
11.1 Trends in Internet Measurement 419
11.2 Difficulties 424
11.3 Future Work 426
11.3.1 Research Challenges 426
11.3.2 Emerging Questions 428
Bibliography 431
Index 473
