# Machine Learning in Python. Essential Techniques for Predictive Analysis

• ID: 3110077
• Book
• 360 Pages
• John Wiley and Sons Ltd
1 of 4

SIMPLE, EFFECTIVE WAY TO ANALYZE DATA AND PREDICT OUTCOMES WITH PYTHON

Machine learning focuses on prediction using what you know to predict what you would like to know based on historical relationships between the two. At its core, it′s a mathematical/algorithm–based technology that, until recently, required a deep understanding of math and statistical concepts, and fluency in R and other specialized languages. Machine Learning in Python simplifies machine learning for a broader audience and wider application by focusing on two algorithm families that effectively predict outcomes, and by showing you how to apply them using the popular and accessible Python programming language.

Author Michael Bowles draws from years of machine learning expertise to walk you through the design, construction, and implementation of your own machine learning solutions. The algorithms are explained in simple terms with no complex math, and sample code is provided to help you get started right away. You′ll delve deep into the mechanisms behind the constructs, and learn how to select and apply the algorithm that will best solve the problem at hand, whether simple or complex. Detailed examples illustrate the machinery with specific, hackable code, and descriptive coverage of linear regression and ensemble methods helps you understand the fundamental processes at work in machine learning. The methods are effective and well tested, and the results speak for themselves.

Designed specifically for those without a specialized math or statistics background, Machine Learning in Python shows you how to:

• Select the right algorithm for the job
• Learn the mechanisms and prepare the data
• Master core Python machine learning packages
• Build versatile predictive models that work
• Apply trained models in practice for various uses
• Measure model performance for better QC and application
• Use provided sample code to design and build your own model
Note: Product cover images may vary from those shown
2 of 4

Introduction xxiii

Chapter 1 The Two Essential Algorithms for Making Predictions 1

Why Are These Two Algorithms So Useful? 2

What Are Penalized Regression Methods? 7

What Are Ensemble Methods? 9

How to Decide Which Algorithm to Use 11

The Process Steps for Building a Predictive Model 13

Framing a Machine Learning Problem 15

Feature Extraction and Feature Engineering 17

Determining Performance of a Trained Model 18

Chapter Contents and Dependencies 18

Summary 20

Chapter 2 Understand the Problem by Understanding the Data 23

The Anatomy of a New Problem 24

Different Types of Attributes and Labels Drive Modeling Choices 26

Classification Problems: Detecting Unexploded Mines Using Sonar 28

Physical Characteristics of the Rocks Versus Mines Data Set 29

Statistical Summaries of the Rocks versus Mines Data Set 32

Visualization of Outliers Using Quantile ]Quantile Plot 35

Statistical Characterization of Categorical Attributes 37

How to Use Python Pandas to Summarize the

Rocks Versus Mines Data Set 37

Visualizing Properties of the Rocks versus Mines Data Set 40

Visualizing with Parallel Coordinates Plots 40

Visualizing Interrelationships between Attributes and Labels 42

Visualizing Attribute and Label Correlations Using a Heat Map 49

Summarizing the Process for Understanding Rocks versus Mines Data Set 50

Real ]Valued Predictions with Factor Variables: How Old Is Your Abalone? 50

Parallel Coordinates for Regression Problems Visualize Variable Relationships for Abalone Problem 56

How to Use Correlation Heat Map for Regression Visualize Pair ]Wise Correlations for the Abalone Problem 60

Real ]Valued Predictions Using Real ]Valued Attributes: Calculate How Your Wine Tastes 62

Multiclass Classification Problem: What Type of Glass Is That? 68

Summary 73

Chapter 3 Predictive Model Building: Balancing Performance, Complexity, and Big Data 75

The Basic Problem: Understanding Function Approximation 76

Working with Training Data 76

Assessing Performance of Predictive Models 78

Factors Driving Algorithm Choices and Performance Complexity and Data 79

Contrast Between a Simple Problem and a Complex Problem 80

Contrast Between a Simple Model and a Complex Model 82

Factors Driving Predictive Algorithm Performance 86

Choosing an Algorithm: Linear or Nonlinear? 87

Measuring the Performance of Predictive Models 88

Performance Measures for Different Types of Problems 88

Simulating Performance of Deployed Models 99

Achieving Harmony Between Model and Data 101

Choosing a Model to Balance Problem Complexity, Model Complexity, and Data Set Size 102

Using Forward Stepwise Regression to Control Overfitting 103

Evaluating and Understanding Your Predictive Model 108

Control Overfitting by Penalizing Regression

Coefficients Ridge Regression 110

Summary 119

Chapter 4 Penalized Linear Regression 121

Why Penalized Linear Regression Methods Are So Useful 122

Extremely Fast Coefficient Estimation 122

Variable Importance Information 122

Extremely Fast Evaluation When Deployed 123

Reliable Performance 123

Sparse Solutions 123

Problem May Require Linear Model 124

When to Use Ensemble Methods 124

Penalized Linear Regression: Regulating Linear Regression for Optimum Performance 124

Training Linear Models: Minimizing Errors and More 126

Adding a Coefficient Penalty to the OLS Formulation 127

Other Useful Coefficient Penalties Manhattan and ElasticNet 128

Why Lasso Penalty Leads to Sparse Coefficient Vectors 129

ElasticNet Penalty Includes Both Lasso and Ridge 131

Solving the Penalized Linear Regression Problem 132

Understanding Least Angle Regression and Its Relationship to Forward Stepwise Regression 132

How LARS Generates Hundreds of Models of Varying Complexity 136

Choosing the Best Model from The Hundreds LARS Generates 139

Using Glmnet: Very Fast and Very General 144

Comparison of the Mechanics of Glmnet and LARS Algorithms 145

Initializing and Iterating the Glmnet Algorithm 146

Extensions to Linear Regression with Numeric Input 151

Solving Classification Problems with Penalized Regression 151

Working with Classification Problems Having More Than Two Outcomes 155

Understanding Basis Expansion: Using Linear Methods on Nonlinear Problems 156

Incorporating Non–Numeric Attributes into Linear Methods 158

Summary 163

Chapter 5 Building Predictive Models Using Penalized Linear Methods 165

Python Packages for Penalized Linear Regression 166

Multivariable Regression: Predicting Wine Taste 167

Building and Testing a Model to Predict Wine Taste 168

Training on the Whole Data Set before Deployment 172

Basis Expansion: Improving Performance by Creating New Variables from Old Ones 178

Binary Classification: Using Penalized Linear Regression to Detect Unexploded Mines 181

Build a Rocks versus Mines Classifier for Deployment 191

Multiclass Classification: Classifying Crime Scene

Glass Samples 204

Summary 209

Chapter 6 Ensemble Methods 211

Binary Decision Trees 212

How a Binary Decision Tree Generates Predictions 213

How to Train a Binary Decision Tree 214

Tree Training Equals Split Point Selection 218

How Split Point Selection Affects Predictions 218

Algorithm for Selecting Split Points 219

Multivariable Tree Training Which Attribute to Split? 219

Recursive Splitting for More Tree Depth 220

Overfitting Binary Trees 221

Measuring Overfit with Binary Trees 221

Balancing Binary Tree Complexity for Best Performance 222

Modifications for Classification and Categorical Features 225

Bootstrap Aggregation: Bagging 226

How Does the Bagging Algorithm Work? 226

Bagging Performance Bias versus Variance 229

How Bagging Behaves on Multivariable Problem 231

Bagging Needs Tree Depth for Performance 235

Summary of Bagging 236

Basic Principle of Gradient Boosting Algorithm 237

Parameter Settings for Gradient Boosting 239

How Gradient Boosting Iterates Toward a Predictive Model 240

Getting the Best Performance from Gradient Boosting 240

Gradient Boosting on a Multivariable Problem 244

Random Forest 247

Random Forests: Bagging Plus Random Attribute Subsets 250

Random Forests Performance Drivers 251

Random Forests Summary 252

Summary 252

Chapter 7 Building Ensemble Models with Python 255

Solving Regression Problems with Python Ensemble Packages 255

Building a Random Forest Model to Predict Wine Taste 256

Constructing a Random Forest Regressor Object 256

Modeling Wine Taste with Random Forest Regressor 259

Visualizing the Performance of a Random

Forests Regression Model 262

Using Gradient Boosting to Predict Wine Taste 263

Using the Class Constructor for Gradient Boosting Regressor 263

Implement a Regression Model 267

Assessing the Performance of a Gradient Boosting Model 269

Coding Bagging to Predict Wine Taste 270

Incorporating Non–Numeric Attributes in Python Ensemble Models 275

Coding the Sex of Abalone for Input to Random Forest Regression in Python 275

Assessing Performance and the Importance of Coded Variables 278

Coding the Sex of Abalone for Gradient Boosting Regression in Python 278

Assessing Performance and the Importance of Coded Variables with Gradient Boosting 282

Solving Binary Classification Problems with Python Ensemble Methods 284

Detecting Unexploded Mines with Python Random Forest 285

Constructing a Random Forests Model to Detect Unexploded Mines 287

Determining the Performance of a Random Forests Classifier 291

Detecting Unexploded Mines with Python Gradient Boosting 291

Determining the Performance of a Gradient Boosting Classifier 298

Solving Multiclass Classification Problems with Python Ensemble Methods 302

Classifying Glass with Random Forests 302

Dealing with Class Imbalances 305

Classifying Glass Using Gradient Boosting 307

Assessing the Advantage of Using Random Forest Base Learners with Gradient Boosting 311

Comparing Algorithms 314

Summary 315

Index 319

Note: Product cover images may vary from those shown
3 of 4