How to Build a Machine Learning Model?
Forward Selection, Backward Elimination, All-in-one, and Bi-Directional Elimination Machine Learning Model.
In Multiple Linear Regression, we cannot take all the variable into account in order to build a machine learning model because it confuses the machine for doing the prediction and the machine gives ambiguous results makes machine difficult to do prediction.
If we fill all the variables into the model then 2 things happen :
- Garbage in, Garbage out
2. It becomes difficult for data scientists to do the explanation.
There are 4 machine learning models:
1. All-in-one
2. Backward elimination
3. Forward selection
4. Bidirectional elimination
All-in-one Model :
In this model, we put all the attributes or columns which helps to do the prediction. This machine learning model is good when independent variables or matrix of features are less.
Backward elimination :
In this machine learning model, we approach stepwise:
Step 1: Select a significance level to stay in the model. The significance level is a measure of the strength of the evidence that must be present in your sample data-set. Significance value 0.05 means that a 5% risk exists or actual difference involved.
Step 2: Fitting of all possible predictors that are all possible attributes of the column.
Step 3: We need to derive the p-value(it is the probability that you would obtain the effect observed in your sample data-set ) from the data-set.
Step 4: If the p-value is less than your significance level then you can prove that the Null Hypothesis is rejected that is your attribute or column is significant for the model, otherwise, we remove the column having a p-value greater than the significant level.
Step 5: Fit the model without that column.
Step 6: Follow steps 3,4 & 5 until p-values of all the attributes or columns are less than a significant level.
Forward Selection:
In this machine learning model, we approach stepwise:
Step 1: Select a significance level to stay in the model(SL=0.05).
Step 2:Start with a Null model. The Null model has no predictors, just one intercept. So, search through all the attributes having the lowest P-value.
Step 3: Keep this variable and fit all possible models with one extra predictor added to the one(s) you have already.
Step 4: Consider all the attributes or predictors with the lowest p-value.
Step 5: If the p-value is less than the significant level then go to Step 3, otherwise the Model is ready.
Bidirectional Elimination:
In this machine learning model, we approach stepwise:
Step 1: Select a significance level to stay in the model(SL=0.05).
Step 2: Perform the next step of “Forward Selection” (new variables must have p-value < Significance Level)
Step 3: Perform All steps of “Backward Elimination” (old variables must have p-value <Significance Level)
Step 4: Repeat Step 2 and Step 3.
Step 5: No new variables can enter and no old variables can exit.
