Figure 1: Effect of using a linear model versus a Generalized Additive Model. Figure and caption from https://towardsdatascience.com/generalised-additive-models-6dfbedf1350a
The Generalized Additive Model (GAM), which was invented by Trevor Hastie and Robert Tibshirani in 1986 (Hastie and Tibshirani, 1986; 1990), is based on a simple conceptual idea: Relationships between the model predictions and the dependent variable follow smooth patterns that can be linear or nonlinear; and these smooth relationships generate the predictions by simply adding them up. Thus, a GAM is an additive modeling technique where the data are fit with smooth functions which, depending on the underlying patterns in the data, can be nonlinear. A GAM can capture common nonlinear patterns that a classic linear model cannot. These patterns range from “hockey sticks” – which occur when you observe a sharp change in the response variable – to various types of “mountain shaped” curves (Figure 1). Moreover, with a GAM the smoothness of the predictor functions can be controlled to prevent overfitting, thus helping to deal with the tradeoff between bias and variance.
The GAM is a powerful yet simple technique that has the interpretability advantages of a Generalized Linear Model (like linear regression) where the contribution of each independent variable to the prediction is clearly encoded. But a GAM has more flexibility because the relationship between independent and dependent variable is not assumed to be linear. A GAM is like Generalized Linear Model that is allowed to learn non-linear features. Moreover, it is not necessary to know a priori what type of predictive functions will be needed. These functions are automatically derived during model estimation. The GAM strikes a balance between the interpretable, yet biased, linear models like linear regression, and the extremely flexible, “black box” learning algorithms like neural networks, i.e., a GAM provides an interpretable model for non-linear data.
References:
Hastie, T. and R. Tibshirani, (1986) Generalized Additive Models, Statistical Science, 1(3): 297-318.
Hastie, T. and R. Tibshirani (1990) Generalized Additive Models, New York: Chapman and Hall.