I spend a bit of time data wrangling. I try to pay mind to what the predictive analytic technique needs. Likewise, it does things on its own too. Then again, when interpreting results, I think on it again. Worse, when I try to compare models or create an ensemble, I really need to know. So, I made this one stop ref.
First of all, it is important to understand how different techniques handle data irregularity. This is a simple post that aggregates some things to know. Let’s compare decision trees, linear regression, and neural networks.
Comparison of data wrangling
| Examples | Decision Trees
| Linear Regression
| Neural Networks
| |
Data Types | Categorical vs continuous. Units of measure. | Continuous vars are binned. | Categorical vars are made continuous. Also, can lessen sizing challenge with transformations. | Categorical vars are made continuous. Also, can adaptive normalize orders-of-magnitude. |
Missing Values | Missing at Random (MAR). Similarly, Missing Completely at Random (MCAR). Not Missing at Random (NMAR). | Doesn’t care, but there are different ways to deal (eg dumping into most popular node or keep as separate bin). Further, can use surrogate splitting rules. | Cannot handle missing values. Thus, must drop or impute. Dropping NMAR can create bias. Generally, there are many ways to impute. | Cannot handle missing values. Thus, must drop or impute. Dropping NMAR can create bias. Also, there are many ways to impute. |
Distributions | Skewness. Outliers. Also, Class imbalance or small disjuncts. | No assumption about inputs or targets distros. Also, skewness can cause problems. | Assumes multivariate normality. Generally, outliers can cause problems. Can do transforms to make normal. | Doesn’t assume any pattern. Also, problems can occur when skewed more than lognormal. |
Unbalanced Data (bias) | Unrepresentative sample. Faulty polling. Awkward binning. Cherry picking. | Overall, is low bias (no assumption about target) and high variance (small input change makes big difference). Also, could change penalties for wrong classification. Or, can limit tree depth. | Can perform regularization (to prevent more complex models). Further, can add a weighting var. | Can create drop out layers (deactivated neurons are temporarily not propagated). Also, can perform regularization to prevent more complex models. |
Variable Relationships | Between vars. Conversely, between predictors and targets. | No assumption (is non-parametric). Further, depth of tree lets target be non-linear. Generally, likes one good var for first split. | Assumes no correlation among vars and predictors-target is linear. Further, makes assumptions about residuals too. Try to combine vars or use PCA. | Can find non-linear relationship of predictors-target. Generally, likes a good a priori starting point. Try to combine vars or use PCA. |
For more info about Perficient and predictive analytics: Data, Cloud, Analytics, Big Data
