Overfitting

 

Geometric explanation for how two object categories are discriminated by a perceptron.

The objects have two features, such as size and brightness, which have values

(x,y) and are plotted on each graph. The two types of objects (pluses and squares) in

the panel on the left can be separated by a straight line that passes between them;

this discrimination can be learned by a perceptron. The two types of objects in the

other two panels cannot be separated by a straight line, but those in the center panel

can be separated by a curved line. The objects in the panel on the right would have to

be gerrymandered to separate the two types. The discriminations in all three panels

could be learned by a deep learning network if enough training data were available.