Generalized Linear Models

In cases where the dependent variable follows a distribution that is different from a normal distribution, we have a technique that generalizes OLS principles. This is known as the generalized linear model (GLM). If the target distribution falls under the *exponential family* of distributions, it uses a link function as a bridge to map the regression. Typically the method of iteratively reweighted least squares is used to fit the model due to quick convergence properties.

- Families
    - Binomial
    - Gaussian
    - Gamma
    - Inverse Gaussian
    - Poisson
- Links
    - Canonical
    - C Log Log
    - Identity
    - Inverse MuSq
    - Logit
    - Log
    - Log Log
    - Probit
    - Reciprocal



For a GLM, three pieces are needed:

  • A distribution belonging to the exponential family
  • A linear predictor
  • A link function that maps the linear predictor to the target distribution

The exponential family of distributions are ones whose density functions can be expressed in the form of:

$$ f_Y(\mathbf{y} \mid \boldsymbol\theta, \tau) = h(\mathbf{y},\tau) \exp \left(\frac{\mathbf{b}(\boldsymbol\theta)^{\rm T}\mathbf{T}(\mathbf{y}) - A(\boldsymbol\theta)} {d(\tau)} \right) $$

We implement the following families:

  • Binomial
  • Gaussian
  • Gamma
  • Inverse Gaussian
  • Poisson

We describe the linear predictor as. $$ \eta = X \beta $$

Then we define a link function that maps this linear predictor to mean of the target distribution. $$ E(Y \mid X) = \mu = g^{-1}(\eta) $$

We implement the following links:

Canonical (will choose the default link for the chosen Family)
    Family: Gaussian     → Link: Identity
    Family: Binomial     → Link: Logit
    Family: Poisson      → Link: Log
    Family: Gamma        → Link: Reciprocal
    Family: Inv Gaussian → Link: Inverse Mu Squared
C Log Log
Inverse Mu Squared
  • coefficient
  • standard error
  • t-statisitc
  • p-value
  • convergence
  • fisher iterations
  • residual null
  • residual deviance