Predicting the Rossmann Store Sales Using Different Models in R --- Kaggle IN class Project

OK, Let us Start our Project ----- Predicting the Sales

Step 0 : Define the Root Mean Square Function for checking my Model in the future.

Step 1 : Loading the Data Set and Combine them correspondingly.

It is Always important to know what kind of data we are dealing with
Use summary(train) to see, and the result is below:

Step 2 : Explorative Data analysis. Take a closer look at the data we have.
Now we could First Make a Huge Graph containing all possible variables, VS Sales ;

It seems that “Customers” has a positive relation with Sales; However, Test Data does not have “Customers”. Under this condition, I think deleting “Customers” in the future modeling will be helpful;
From the Graph, it is easy to see that most of our features are categorical features;
There are many interesting points I noticed. First, The Sales will be greater on weekday than weekend, as we could see that Sunday has the lowest Sales; Second, Holiday will influence Sales a lot.
How About other Variables ? Let us see whether “StoreType” has anything interesting.

It seems that “StoreType a “ will have better possibility to get higher sales.
Moreover, check “CompetitionDistance” with “StoreType”:

The Plot tells us that Higher Sales dependent on Smaller Distance with StoreType “a”; So, ‘CompetitionDistance’, “StoreType” will be crucial features for modeling.

Step 3 : Transforming the Data
We could define “week”, “Day”, “Month” using “Date”;

We could make Dummy Variables using “StateHoliday”, “StoreType”;
Also, we could create “WeekEnd” using “DayOfWeek”;
According to any possible holidays, we combine them into “HolidayT”;
Using “HolidayT” and “CompetitionDistance”, create “DistanceHoliday”;

Next, we need to delete some useless variables;

Then we have transformed our data to be available in Modeling !

Step 4 : Modeling
As we do not want “Customers”, and we need to create data set suitable to different models;

The First Model : Linear Regression + Lasso Regression + Ridge Regression

For Linear Regression Model, we could use StepWise Method to select the variables;

Then we could combine these three regression model together with different weights, and we could get our first Kaggle Score !

The Second Model : Random Forest
Before Using the Raw Model, we need to select our best parameters;

Applying the Random Forest Model to the test file,
I Tried many times with Random Forest Model, sometimes with mtry = 10, sometimes with mtry = 9;
And Below is the best Kaggle Score using Random Forest Model;

The Third Model : XgBoost
Similarly, before training the model, we need to select the best parametes;
Here, I used grid-search for several parametes;

Then We apply the xgboost model with the best parameters ;

Using similar codes shown in Regression Model Session, I get the result from xgboost and submit on Kaggle to get Score.
Through several attempts, the best Score is :

The Final Model : Combined with Random Forest and Xgboost
We could Try some combination between models. The usage of our Model 1 (combination of regressions) may be helpful in this time.
Through several attempts, I found that 0.75 xgboost + 0.25 random_forest will be the best combination under my data.

Then the final and the best Kaggle Score is the combination of random forest and Xgboost;

End of the Project, we used several models to predict the sales;
Lasso Regression, Ridge Regression, LInear Regression, Random Forest, Xgboost, And we apply some combinations;
Here is the rank of my result out of 80 students ! Hope this is a good help to anyone want to learn machine learning.