+353-1-416-8900REST OF WORLD
+44-20-3973-8888REST OF WORLD
1-917-300-0470EAST COAST U.S
1-800-526-8630U.S. (TOLL FREE)

Big Data. Concepts, Technology, and Architecture. Edition No. 1

  • Book

  • 368 Pages
  • June 2021
  • John Wiley and Sons Ltd
  • ID: 5837899

Learn Big Data from the ground up with this complete and up-to-date resource from leaders in the field 

Big Data: Concepts, Technology, and Architecture delivers a comprehensive treatment of Big Data tools, terminology, and technology perfectly suited to a wide range of business professionals, academic researchers, and students. Beginning with a fulsome overview of what we mean when we say, “Big Data,” the book moves on to discuss every stage of the lifecycle of Big Data. 

You’ll learn about the creation of structured, unstructured, and semi-structured data, data storage solutions, traditional database solutions like SQL, data processing, data analytics, machine learning, and data mining. You’ll also discover how specific technologies like Apache Hadoop, SQOOP, and Flume work. 

Big Data also covers the central topic of big data visualization with Tableau, and you’ll learn how to create scatter plots, histograms, bar, line, and pie charts with that software. 

Accessibly organized, Big Data includes illuminating case studies throughout the material, showing you how the included concepts have been applied in real-world settings. Some of those concepts include: 

  • The common challenges facing big data technology and technologists, like data heterogeneity and incompleteness, data volume and velocity, storage limitations, and privacy concerns 
  • Relational and non-relational databases, like RDBMS, NoSQL, and NewSQL databases 
  • Virtualizing Big Data through encapsulation, partitioning, and isolating, as well as big data server virtualization 
  • Apache software, including Hadoop, Cassandra, Avro, Pig, Mahout, Oozie, and Hive 
  • The Big Data analytics lifecycle, including business case evaluation, data preparation, extraction, transformation, analysis, and visualization 

Perfect for data scientists, data engineers, and database managers, Big Data also belongs on the bookshelves of business intelligence analysts who are required to make decisions based on large volumes of information. Executives and managers who lead teams responsible for keeping or understanding large datasets will also benefit from this book. 

 

Table of Contents

Acknowledgments xi

About the Author xii

1 Introduction to the World of Big Data 1

1.1 Understanding Big Data 1

1.2 Evolution of Big Data 2

1.3 Failure of Traditional Database in Handling Big Data 3

1.4 3 Vs of Big Data 4

1.5 Sources of Big Data 7

1.6 Different Types of Data 8

1.7 Big Data Infrastructure 11

1.8 Big Data Life Cycle 12

1.9 Big Data Technology 18

1.10 Big Data Applications 21

1.11 Big Data Use Cases 21

Chapter 1 Refresher 24

2 Big Data Storage Concepts 31

2.1 Cluster Computing 32

2.2 Distribution Models 37

2.3 Distributed File System 43

2.4 Relational and Non-Relational Databases 43

2.5 Scaling Up and Scaling Out Storage 47

Chapter 2 Refresher 48

3 NoSQL Database 53

3.1 Introduction to NoSQL 53

3.2 Why NoSQL 54

3.3 CAP Theorem 54

3.4 ACID 56

3.5 BASE 56

3.6 Schemaless Databases 57

3.7 NoSQL (Not Only SQL) 57

3.8 Migrating from RDBMS to NoSQL 76

Chapter 3 Refresher 77

4 Processing, Management Concepts, and Cloud Computing 83

Part I: Big Data Processing and Management Concepts 83

4.1 Data Processing 83

4.2 Shared Everything Architecture 85

4.3 Shared-Nothing Architecture 86

4.4 Batch Processing 88

4.5 Real-Time Data Processing 88

4.6 Parallel Computing 89

4.7 Distributed Computing 90

4.8 Big Data Virtualization 90

Part II: Managing and Processing Big Data in Cloud Computing 93

4.9 Introduction 93

4.10 Cloud Computing Types 94

4.11 Cloud Services 95

4.12 Cloud Storage 96

4.13 Cloud Architecture 101

Chapter 4 Refresher 103

5 Driving Big Data with Hadoop Tools and Technologies 111

5.1 Apache Hadoop 111

5.2 Hadoop Storage 114

5.3 Hadoop Computation 119

5.4 Hadoop 2.0 129

5.5 HBASE 138

5.6 Apache Cassandra 141

5.7 SQOOP 141

5.8 Flume 143

5.9 Apache Avro 144

5.10 Apache Pig 145

5.11 Apache Mahout 146

5.12 Apache Oozie 146

5.13 Apache Hive 149

5.14 Hive Architecture 151

5.15 Hadoop Distributions 152

Chapter 5 Refresher 153

6 Big Data Analytics 161

6.1 Terminology of Big Data Analytics 161

6.2 Big Data Analytics 162

6.3 Data Analytics Life Cycle 166

6.4 Big Data Analytics Techniques 170

6.5 Semantic Analysis 175

6.6 Visual analysis 178

6.7 Big Data Business Intelligence 178

6.8 Big Data Real-Time Analytics Processing 180

6.9 Enterprise Data Warehouse 181

Chapter 6 Refresher 182

7 Big Data Analytics with Machine Learning 187

7.1 Introduction to Machine Learning 187

7.2 Machine Learning Use Cases 188

7.3 Types of Machine Learning 189

Chapter 7 Refresher 196

8 Mining Data Streams and Frequent Itemset 201

8.1 Itemset Mining 201

8.2 Association Rules 206

8.3 Frequent Itemset Generation 210

8.4 Itemset Mining Algorithms 211

8.5 Maximal and Closed Frequent Itemset 229

8.6 Mining Maximal Frequent Itemsets: the GenMax Algorithm 233

8.7 Mining Closed Frequent Itemsets: the Charm Algorithm 236

8.8 CHARM Algorithm Implementation 236

8.9 Data Mining Methods 239

8.10 Prediction 240

8.11 Important Terms Used in Bayesian Network 241

8.12 Density Based Clustering Algorithm 249

8.13 DBSCAN 249

8.14 Kernel Density Estimation 250

8.15 Mining Data Streams 254

8.16 Time Series Forecasting 255

9 Cluster Analysis 259

9.1 Clustering 259

9.2 Distance Measurement Techniques 261

9.3 Hierarchical Clustering 263

9.4 Analysis of Protein Patterns in the Human Cancer-Associated Liver 266

9.5 Recognition Using Biometrics of Hands 267

9.6 Expectation Maximization Clustering Algorithm 274

9.7 Representative-Based Clustering 277

9.8 Methods of Determining the Number of Clusters 277

9.9 Optimization Algorithm 284

9.10 Choosing the Number of Clusters 288

9.11 Bayesian Analysis of Mixtures 290

9.12 Fuzzy Clustering 290

9.13 Fuzzy C-Means Clustering 291

10 Big Data Visualization 293

10.1 Big Data Visualization 293

10.2 Conventional Data Visualization Techniques 294

10.3 Tableau 297

10.4 Bar Chart in Tableau 309

10.5 Line Chart 310

10.6 Pie Chart 311

10.7 Bubble Chart 312

10.8 Box Plot 313

10.9 Tableau Use Cases 313

10.10 Installing R and Getting Ready 318

10.11 Data Structures in R 321

10.12 Importing Data from a File 335

10.13 Importing Data from a Delimited Text File 336

10.14 Control Structures in R 337

10.15 Basic Graphs in R 341

Index 347

Authors

Balamurugan Balusamy Galgotias University, Greater Noida, India. Nandhini Abirami R VIT University, Vellore, India. Seifedine Kadry Noroff University College, Kristiansand, Norwa. Amir H. Gandomi University of Technology Sydney, Australia.