Skip to main content

​
📈 Linear Regression

Linear regression models the relationship between a dependent variable yy and one or more features x1,…,xnx_1, \dots, x_n by fitting a linear function.

​
Hypothesis

For an input vector x=[1,x1,…,xn]T\mathbf{x} = [1, x_1, \dots, x_n]^T, the model predicts y^=xTθ=θ0+θ1x1+⋯+θnxn.\hat{y} = \mathbf{x}^T \boldsymbol{\theta} = \theta_0 + \theta_1 x_1 + \cdots + \theta_n x_n.

​
Loss Function

Parameters θ\boldsymbol{\theta} are learned by minimizing the mean squared error (MSE): J(θ)=1m∑i=1m(y^(i)−y(i))2.J(\boldsymbol{\theta}) = \frac{1}{m} \sum_{i=1}^{m} (\hat{y}^{(i)} - y^{(i)})^2.

​
Example (scikit-learn)

import numpy as np
from sklearn.linear_model import LinearRegression

# toy dataset
X = np.array([[1], [2], [3], [4]])
y = np.array([2, 3, 3, 5])

model = LinearRegression()
model.fit(X, y)

pred = model.predict([[5]])
print(pred)

​
Interpretation

  • Coefficients θj\theta_j show how much the prediction changes per unit increase in feature xjx_j.
  • The model is fast and interpretable but assumes a linear relationship between features and target.