+353-1-416-8900REST OF WORLD
+44-20-3973-8888REST OF WORLD
1-917-300-0470EAST COAST U.S
1-800-526-8630U.S. (TOLL FREE)

Ensemble Classification Methods with Applications in R. Edition No. 1

  • Book

  • 224 Pages
  • October 2018
  • John Wiley and Sons Ltd
  • ID: 5225237

An essential guide to two burgeoning topics in machine learning - classification trees and ensemble learning 

Ensemble Classification Methods with Applications in R introduces the concepts and principles of ensemble classifiers methods and includes a review of the most commonly used techniques. This important resource shows how ensemble classification has become an extension of the individual classifiers. The text puts the emphasis on two areas of machine learning: classification trees and ensemble learning. The authors explore ensemble classification methods’ basic characteristics and explain the types of problems that can emerge in its application.

Written by a team of noted experts in the field, the text is divided into two main sections. The first section outlines the theoretical underpinnings of the topic and the second section is designed to include examples of practical applications. The book contains a wealth of illustrative cases of business failure prediction, zoology, ecology and others. This vital guide:

  • Offers an important text that has been tested both in the classroom and at tutorials at conferences
  • Contains authoritative information written by leading experts in the field
  • Presents a comprehensive text that can be applied to courses in machine learning, data mining and artificial intelligence 
  • Combines in one volume two of the most intriguing topics in machine learning: ensemble learning and classification trees

Written for researchers from many fields such as biostatistics, economics, environment, zoology, as well as students of data mining and machine learning, Ensemble Classification Methods with Applications in R puts the focus on two topics in machine learning: classification trees and ensemble learning.

 

Table of Contents

List of Contributors ix

List of Tables xi

List of Figures xv

Preface xvii

1 Introduction 1
Esteban Alfaro, Matías Gámez, and Noelia García

1.1 Introduction 1

1.2 Definition 1

1.3 Taxonomy of Supervised Classification Methods 2

1.4 Estimation of the Accuracy of a Classification System 3

1.4.1 The Apparent Error Rate 4

1.4.2 Estimation of the True Error Rate 4

1.4.3 Error Rate Estimation Methods 4

1.4.4 The Standard Error 6

1.5 Classification Trees 7

1.5.1 Classification Tree Building 8

1.5.2 Splitting Rule 9

1.5.3 Splitting Criteria 10

1.5.4 Goodness of a Split 10

1.5.5 The Impurity of a Tree 11

1.5.6 Stopping Criteria 11

1.5.7 Overfitting in Classification Trees 12

1.5.8 Pruning Rules 14

2 Limitation of the Individual Classifiers 19
Esteban Alfaro, Matías Gámez, and Noelia García

2.1 Introduction 19

2.2 Error Decomposition: Bias and Variance 20

2.3 Study of Classifier Instability 23

2.4 Advantages of Ensemble Classifiers 26

2.5 Bayesian Perspective of Ensemble Classifiers 28

3 Ensemble Classifiers Methods 31
Esteban Alfaro, Matías Gámez, and Noelia García

3.1 Introduction 31

3.2 Taxonomy of Ensemble Methods 32

3.2.1 Non-Generative Methods 33

3.2.2 Generative Methods 33

3.3 Bagging 34

3.4 Boosting 36

3.4.1 AdaBoost Training Error 40

3.4.2 AdaBoost and the MarginTheory 41

3.4.3 Other Boosting Versions 43

3.4.4 Comparing Bagging and Boosting 46

3.5 Random Forests 46

4 Classification with Individual and Ensemble Trees in R 51
Esteban Alfaro, Matías Gámez, and Noelia García

4.1 Introduction 51

4.2 adabag: An R Package for Classification with Boosting and Bagging 52

4.2.1 The bagging, predict.bagging, and bagging.cv Functions 56

4.2.2 The boosting, predict.boosting, and boosting.cv Functions 65

4.2.3 The margins, plot.margins, errorevol and plot.errorevol Functions 71

4.2.4 The MarginOrderedPruning.Bagging Function 75

4.3 The “German Credit” Example 79

4.3.1 Classification Tree 81

4.3.2 Combination using Bagging 85

4.3.3 Combination using Boosting 88

4.3.4 Combination using Random Forest 90

4.3.5 Cross-Validation Comparison 95

5 Bankruptcy Prediction Through Ensemble Trees 97
Esteban Alfaro, Matías Gámez, and Noelia García

5.1 Introduction 97

5.2 Problem Description 97

5.3 Applications 99

5.3.1 The Dichotomous Case 99

5.3.2 TheThree-Class Case 111

5.4 Conclusions 117

6 Experiments with Adabag in Biology Classification Tasks 119
M. Fernández-Delgado, E. Cernadas, and M. Pérez-Ortiz

6.1 Classification of Color Texture Feature Patterns Extracted From Cells in Histological Images of Fish Ovary 119

6.2 Direct Kernel Perceptron: Ultra-Fast Kernel ELM-Based Classification with Non-Iterative Closed-Form Weight Calculation 122

6.3 Do We Need Hundreds of Classifiers to Solve Real-World Classification Problems? 125

6.4 On the use of Nominal and Ordinal Classifiers for the Discrimination of Stages of Development in Fish Oocytes 129

7 Generalization Bounds for Ranking Algorithms 135
W. Rejchel

7.1 Introduction 135

7.2 Assumptions, Main Theorem, and Application 136

7.3 Experiments 138

7.4 Conclusions 139

8 Classification and Regression Trees for Analyzing Irrigation Decisions 141
S. Andriyas andM.McKee

8.1 Introduction 141

8.2 Theory 143

8.3 Case Study and Methods 144

8.3.1 Study Site and Data Available 144

8.3.2 Model, Specifications, and Performance Evaluation 146

8.4 Results and Discussion 147

8.5 Conclusions 153

9 Boosted Rule Learner and its Properties 155
M. Kubus

9.1 Introduction 155

9.2 Separate-and-Conquer 156

9.3 Boosting in Rule Induction 157

9.4 Experiments 158

9.5 Conclusions 161

10 Credit Scoring with Individuals and Ensemble Trees 163
M. Chrzanowska, E. Alfaro, and D.Witkowska

10.1 Introduction 163

10.2 Measures of Accuracy 164

10.3 Data Description 165

10.4 Classification of Borrowers Applying Ensemble Trees 168

10.5 Conclusions 173

11 An Overview of Multiple Classifier Systems Based on Generalized Additive Models 175
K.W. De Bock, K. Coussement, and D. Cielen

11.1 Introduction 175

11.2 Multiple Classifier Systems Based on GAMs 176

11.2.1 Generalized AdditiveModels 176

11.2.2 GAM-Based Multiple Classifier Systems 177

11.2.3 GAMensPlus: Extending GAMens for Advanced Interpretability 179

11.3 Experiments and Applications 180

11.3.1 A Multi-Domain Benchmark Study of GAM-Based Ensemble Classifiers 180

11.3.2 Benchmarking GAM-Based Ensemble Classifiers in Predictive Customer Analytics 181

11.3.3 A Case Study of GAMensPlus used for Customer Churn Prediction in Financial Services 183

11.4 Software Implementation in R: the GAMens Package 185

11.5 Conclusions 185

References 187

Index 197

Authors

Esteban Alfaro Matías Gámez Noelia García