Big Data Analytics with Spark

Chapter 1: Big Data Technology Landscape (30 pages) Chapter Goal: Familiarize readers with important big data technologies that get used along with Spark. Knowledge of some of these technologies is required in order to effectively use Spark. * Cluster computing - Hadoop MapReduce, HDFS, Hive* Data serialization - Avro, Proto Buffer* Columnar storage - Parquet* Messaging system - Kafka, ZeroMQ* NoSQL databases - HBase, Cassandra* Distributed SQL Query engine - Apache Drill, Impala, PrestoDB Chapter 2: Functional Programming in Scala (30 pages) Chapter Goal: Introduce Scala so that readers can understand and write Spark applications in Scala, which is the primary language supported by Spark. * Key functional programming concepts* Basic Scala constructs* Scala Shell* Simple Build Tool (SBT)* Developing and running a stand-alone Scala app Chapter 3: Spark's Essentials (35 pages) Chapter Goal: Introduce Spark fundamentals and key concepts * What is Spark* Why Spark is hot* Why Spark is faster than Hadoop MapReduce* Resilient Distributed Datasets (RDD)Chapter 4: Spark Shell (10 pages) Chapter Goal: Introduce Spark Shell and show how it can be used for interactive data analysis * Spark shell introduction* Interactive data analysis in Spark-shellChapter 5: A Stand-alone Spark Application (10 pages) Chapter Goal: Provide step-by-step directions for writing and running a Spark application * Basic structure of a stand-alone Spark application* Compiling a Spark applicationChapter 6: Spark Streaming (10 pages) Chapter Goal: Introduce Spark streaming and show an example app using Spark streaming 1. Spark streaming introduction 2. How Spark streaming works 3. A spark streaming example app Chapter 7: Spark SQL (20 pages) Chapter Goal: Introduce Spark SQL along with a few examples * Spark SQL introduction* Why use Spark SQL* Language integrated queries* A sample Spark SQL app* Spark SQL Thrift serverChapter 8: MLib (40 pages) Chapter Goal: Introduce machine learning and MLlib along with a few examples * Machine learning introduction* Linear regression* Logistic regression* Classification* Clustering* Recommender system* Building a machine learning application with Mlib* MLBaseChapter 9: GraphX (10 pages) Chapter Goal: Introduce Graph analysis and GraphX along with a few examples * GraphX introduction* A sample appChapter 10: Deploying Spark (15 pages) This chapter would walk us through deployment of Spark with different cluster management technologies such as YARN, Mesos, and services like AWS (EC2) Deploying Spark Cluster in Standalone Mode (2 pages) Goal: Introduce Standalone Spark cluster manager and provide step-by-step directions for installing and running Spark application on a Standalone Spark cluster * Spark stand-alone cluster introduction* Installing a Spark cluster in stand-alone mode* Running a Spark app on a stand-alone Spark cluster Deploying Spark Cluster with YARN (2 pages) Goal: Introduce YARN cluster manager and provide step-by-step directions for installing and running Spark application on a YARN cluster * YARN introduction* Installing a Spark cluster on YARN* Running a Spark app on a Spark-on-YARN cluster Deploying Spark Cluster with Mesos (2 pages) Goal: Introduce Mesos cluster manager and provide step-by-step directions for installing and running Spark application on a Mesos cluster * Mesos introduction* Installing a Spark cluster with Mesos* Running a Spark app on a Spark-on-Mesos cluster Deploying Spark on AWS EC2 (2 pages) Goal: Provide step-by-step directions for installing and running a Spark cluster on EC2 * EC2 introduction* How to install and run Spark on EC2Chapter 11: Monitoring a Spark Cluster (20 pages) Chapter Goal: Introduce built-in monitoring and management capabilities provided by Spark * Spark management interface* Monitoring a Spark cluster using a web browser* Monitoring Spark libraries and application* Monitoring a Spark cluster using Ganglia