Back to Curriculum
Basic•Machine Learning
Linear Regression
Predicting continuous values using a linear relationship.
Interactive Playground
Initializing Interactive Playground...
Research-Level Deep Dive & Equations
In the mathematical modeling of relationships between variables, the core objective of regression analysis is to characterize the conditional expectation of a scalar response variable given a vector of predictors . We assume that the observed data arises from a true, underlying data-generating process (DGP):
where is the true population regression function, representing the systematic component of , and is an unobserved, random error term representing irreducible noise. The error term satisfies the strict exogeneity condition:
which implies that the noise has zero mean and is completely uncorrelated with the predictors.
In the linear regression framework, we restrict the functional form of to the parametric family of affine functions. Thus, we model the conditional expectation as:
where is the vector of population parameters (including the intercept), and the predictor vector is augmented with a constant as its first element, i.e., .
For a sample of independent and identically distributed (i.i.d.) observations, we stack the scalar equations to write the system in compact matrix notation:
where is the vector of responses, is the design matrix, is the parameter vector, and is the vector of disturbances. We assume the standard homoscedasticity and spherical error structure:
where is the constant noise variance, and is the identity matrix. This spherical error structure assumes that the disturbances are uncorrelated across observations: for all .
Key Equations
Test Your Knowledge
Check whether you have mastered this concept with a quick quiz.
Was this lesson helpful?
Your feedback helps us continuously improve the curriculum and interactive visualizations.