Meditations and Learnings

Meditations and Learnings

Overfitting

Overfitting, a statistical concept, is when a model is too sensitive to the data it’s been fed, and therefore stops being generalisable. It’s learning something too well. For instance, artificial neural networks have a training data set: the data that they learn from. All training sets are finite, and often the data comes from the same source and is highly correlated in some non-obvious way. Because of this, artificial neural networks are in constant danger of becoming overfitted. When a network becomes overfitted, it will be good at dealing with the training data set but will fail at other similar data sets. All learning is basically a tradeoff between specificity and generality in this manner.