Classification Models

Students will learn about ZeroR, OneR and Naive Bayesian classification algorithms

Decision tree is the most used classification algorithm. Decision tree builds classification or regression models in the form of a tree structure. It breaks down a dataset into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. The final result is a tree with decision nodes and leaf nodes.

Linear Discriminant Analysis (LDA) is based upon the concept of searching for a linear combination of the variables that best separates two classes. LDA is originally developed in 1936 by R. A. Fisher. It is simple, mathematically robust and often produces models whose accuracy is as good as more complex methods.

Logistic Regression is a classification model which predicts the probability of an outcome that can only have two values (e.g., binary), Logistic regression produces a logistic curve, which is limited to values between 0 and 1. Logistic regression is similar to a linear regression, but the curve is constructed using the natural logarithm of the “odds” of the target variable, rather than the probability. Moreover, the predictors do not have to be normally distributed or have equal variance in each group.

K nearest neighbors is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure (e.g., distance functions). KNN has been used in statistical estimation and pattern recognition already in the beginning of 1970’s as a non-parametric technique.

An artificial neutral network (ANN) is a system that is based on the biological neural network, such as the brain. The ANN attempts to recreate the computational mirror of the biological neural network, although it is not comparable since the number and complexity of neurons and the used in a biological neural network is many times more than those in an artificial neutral network.

Model Evaluation is an integral part of the model development process. It helps to find the best model that represents our data and how well the chosen model will work in the future. Evaluating model performance with the data used for training is not acceptable in data mining because it can easily generate overoptimistic and overfitted models. There are two methods of evaluating models in data mining, Hold-Out and Cross-Validation. To avoid overfitting, both methods use a test set (not seen by the model) to evaluate model performance.

Students will learn how to build a classification model to predict the probability of default for small businesses.

The second part of the sample project is about Bivariate Data Exploration. Students learn how to use different visualization methods to demonstrate the relationship between categorical and numerical variables.

The third part of the sample project is about building classification models such as ZeroR, OneR, Bayesian, Decision Tree and more.

The forth part is about models evaluation using confusion matrix, lift or gain chart, and ROC chart. Students also learn how to deploy a classification model.