Integrated learning based on popular portal
integration modeling is a powerful way to improve the performance of the model.This article as a popular introductory tutorial, the integration is introduced through an example of rapid learning basic knowledge of how to actually get different learning modules, and emphatically discussed several integration technology is widely used in the industry, including Bagging, Boosting, Stacking, etc.
An introduction to
Integrated modeling is a powerful way to improve the performance of the model.On you may build models using the integrated study is usually effective.Time and time again, people use integration model in Kaggle such competition, and benefit from it.
Integrated learning is a wide range of topics, wide to beyond your imagination.For the purpose of this article, I will cover the basic concepts of integrated modeling and thoughts.This should be enough to make you own machine start integration model is established.As usual, we try to make things as simple as possible.
Let us through an example to quickly understand the basis of integrated learning knowledge.This example will be out of a day is how to use integrated in unsuspecting case study.
Case: I want to invest in a company XYZ.I don't know its performance.So I want to someone give me some advice, to see if the company's stock price will increase more than 6% per year.I'm going to communicate with have different experience in the field.
1, XYZ company clerk: this person know internal workings, and know the inside information of the company.But he doesn't know the competitor's innovation situation, how will the technology development, and this will have any impact on the XYZ company product development.In the past, he has 70% of the time is right.
2, XYZ company financial advisor: this person know very well in the fierce competition environment of the company's strategy will be how to effect.However, he is not the result of the company's internal policy what will understand.In the past, he has 75% of the time is right.
3, the stock market traders, who for the past three years have been looking at the company's shares.He knows the cyclical trends, as well as the overall performance of the stock market.He also formed over how the stock may change over time and a strong intuition.In the past, he has 70% of the time is right.
4, a competitor's clerk: this person know the competition of the company's internal workings, and realised that has changed.His lack of focus on the company's understanding, and poor awareness of competitors related to external factors.In the past, he has 60% of the time is right.
5, the same field of market research team: the team will analyze the XYZ company products and other products of user experience, and this is how to change over time.Because they are dealing with customers, and based on their own goals, they don't know what will happen to the XYZ company.In the past, when they have 75% of the judgment is right.
6, social media experts: who can help us understand how XYZ company product market positioning to them.And over time, how was your clients to this company.In addition to the field of digital marketing, he isn't too concerned about the details of the other fields.In the past, he has 65% of the time is right.
Through all aspects of understanding, we can combine all the information, and make a wise decision.
If all six experts/team think this is a very good decision (assuming all forecasts are independent of each other), we will get combination accuracy:
1-30% * 25% * 30% * 40% * 25% * 35%
1-0.07875 = = 99.92125%
Hypothesis: here's prediction is completely independent of all assume a bit extreme, because they expected.But, we can see that combine different forecast we will grasp how much of a problem.
Now to change the scenario.This time, we have six experts, they are all of the XYZ company employees, and all work in the same department.Everyone has about 70% accuracy.
If we combine these Suggestions, will get more than 99% accuracy?
Obviously not, because of the prediction is made on the similar information set.They will be affected by the similar information set, and they suggest that the only difference is that everyone in the company have a different view.
Stop and think: you're what conclusion from this case?Whether very deep?Write down your opinion in the comments section.
What is the integration of learning?
Integration is the combination of different learning modules (single) model to enhance the stability of the model and the ability to predict.In the example above, we will be together all prediction approach called the integrated learning.
In this article, we will discuss the integration technology of several widely used in the industry.Before us about technology, let's learn how to really get different learning modules.Model will be different because of various reasons, from the training sample data set of the model to the model constructor will result in differences.
The following is the main factor that lead to model different four.The combination of these factors also may lead to different models:
1, different kinds
2, different assumptions
3, different modeling techniques
4, initialization parameters are different
Integrated modeling of error (variance vs. offset)
Error in any model can be resolved mathematically into three components.As follows:
In the present circumstances why this is important?In order to understand what is happening behind integration model, we first need to understand what caused the error in the model.We will briefly introduce the error, and then analyze each integrated learning module.
Offset error between the actual and estimated values are used to measure the average difference, high bias error means that our poor performance of the model being lost important trend.
Variance is measured based on the same observation value, the difference between predicted value.High variance model fitting on your training set, and didn't perform well in training outside of any observation.Below will make you more understand (assuming red dot is the real value, dot is predicted) :
Figure since: Scott Fortman
Usually, when you increase the complexity of the model, due to the low bias model, you will find that the error decreases.But that only happens at a particular point.When you continue to increase the complexity of model, the model will eventually be fitting, so the model began to appear high variance.
A good model should keep a balance between these two kinds of error.This is called the bias variance compromise management.Integration of a way to learn is to perform compromise measure.
Figure since: Scott Fortman
Some commonly used integrated learning technology
1, Bagging: Bagging is trying to achieve similar learning module on small sample set, then the predictive value of averaging.For Bagging in general, you can use different learning modules on different data sets.As you might expect, this can help us reduce the variance.
2, Boosting: Boosting is an iterative technique, based on the classification of it in the last adjustment weights of observations.If the observed value is wrong classification, it will increase the weight of the observed value, and vice versa.Boosting tend to reduce the bias error and then build a powerful prediction model.But sometimes they also on the training data fitting.
3, Stacking: use it to combination model is an interesting way.Here we use a learning
module and output from different learning modules together.Depends on the learning module
we use.Doing so can reduce the bias and variance error.