Computational Methods for Corpus Annotation and Analysis

Preface.- Chapter 1 Introduction. 1.1 Objectives and Rationale of the Book. 1.2 Why Do We Need to Go Beyond Raw Corpora. 1.3 What is Corpus Annotation. 1.4 Organization of the Book.- Chapter 2 Text Processing with the Command Line Interface. 2.1 The Command Line Interface. 2.2 Basic Commands. 2.2.1 Notational Conventions. 2.2.2 Printing the Current Working Directory. 2.2.3 Listing Files and Subdirectories. 2.2.4 Making New Directories. 2.2.5 Changing Directory Locations. 2.2.6 Creating and Editing Text Files with UTF-8 Encoding. 2.2.7 Viewing, Renaming, Moving, Copying, and Removing Files. 2.2.8 Copying, Moving, and Removing Directories. 2.2.9 Using Shell Meta-Characters for File Matching. 2.2.10 Manual Pages, Command History, and Command Line Completion. 2.3 Tools for Text Processing. 2.3.1 Searching for a String with egrep. 2.3.2 Regular Expressions. 2.3.3 Character Translation with tr. 2.3.4 Editing Files from the Command Line with sed. 2.3.5 Data Filtering and Manipulation Using awk. 2.3.6 Task Decomposition and Pipes. 2.4 Summary.- Chapter 3 Lexical Annotation. 3.1 Part-of-Speech Tagging. 3.1.1 What is Part-of-Speech Tagging. 3.1.2 Understanding Part-of-Speech Tagsets. 3.1.3 The Stanford Part-of-Speech Tagger. 3.2 Lemmatization. 3.2.1 What is Lemmatization and Why is it Useful. 3.2.2 The TreeTagger. 3.3 Additional Tools. 3.3.1 The Stanford Tokenizer. 3.3.2 The Stanford Word Segmenter for Arabic and Chinese. 3.3.3 The CLAWS Tagger for English. 3.3.4 The Morpha Lemmatizer for English. 3.4 Summary.- Chapter 4 Lexical Analysis. 4.1 Frequency Lists. 4.1.1 Working with Output Files from the TreeTagger. 4.1.2 Working with Output Files from the Stanford POS Tagger and Morpha. 4.1.3 Analyzing Frequency Lists with Text Processing Tools. 4.2 N-grams. 4.3 Lexical Richness. 4.3.1 Lexical Density. 4.3.2 Lexical Variation. 4.3.3 Lexical Sophistication. 4.3.4 Tools for Lexical Richness Analysis. 4.4 Summary.- Chapter 5 Syntactic Annotation. 5.1 Syntactic Parsing Overview. 5.1.1 What is Syntactic Parsing and Why is it Useful. 5.1.2 Phrase Structure Grammars. 5.1.3 Dependency Grammars. 5.2 Syntactic Parsers. 5.2.1 The Stanford Parser. 5.2.2 Collins' Parser. 5.3 Summary.- Chapter 6 Syntactic Analysis. 6.1 Querying Syntactically Parsed Corpora. 6.1.1 Tree Relationships. 6.1.2 Tregex. 6.2 Syntactic Complexity Analysis. 6.2.1 Measures of Syntactic Complexity. 6.2.2 Syntactic Complexity Analyzers. 6.3 Summary.- Chapter 7 Semantic, Pragmatic and Discourse Analysis.- 7.1 Semantic Field Analysis. 7.1.1 The UCREL Semantic Analysis System. 7.1.2 Profile in Semantics-Lexical in Computerized Profiling. 7.2 Analysis of Propositions. 7.2.1 Computerized Propositional Idea Density Rater. 7.2.2 Analysis of Propositions in Computerized Profiling. 7.3 Conversational Act Analysis in Computerized Profiling. 7.4 Coherence and Cohesion Analysis in Coh-Metrix. 7.5 Text Structure Analysis. 7.6 Summary.- Chapter 8 Summary and Outlook. 8.1 Summary of the Book. 8.2 Future Directions in Computational Corpus Analysis. 8.2.1 Computational Analysis of Language Meaning and Use. 8.2.2 Computational Analysis of Learner Language. 8.2.3 Computational Analysis Based on Specific Language Theories.- Appendix