线性回归的普通最小二乘估计
本文推导了线性回归的普通最小二乘估计量的矩阵形式，并在一元线性回归的情境下给出了求和形式的表达式。
$$
Y=X \widehat{\beta}+e
$$
\[
\beta^{O L S}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y
\]
在一元线性回归的情境下：
\[
\beta_1^{O L S} =\frac{\overline{X Y}-\overline{X} * \overline{Y}}{\overline{X^2}-\left(\overline{X}\right)^2}
\]
\[
\beta_0^{O L S} =\frac{\overline{X^2} * \overline{Y}-\overline{X} * \overline{X Y}}{\overline{X^2}-\left(\overline{X}\right)^2}
\]
线性模型
若我们想要用\(X_k \left(k=1,\ldots,K\right)\)来解释\(Y\)，且\(Y\)关于\(X\)是线性的，即：
$$
Y_t=\beta_0+\beta_1 X_{t 1}+\beta_2 X_{t 2} +\cdots+\beta_K X_{t K}
$$
Note
这里加入了截距项（或偏置项），因此需要加入\(\beta_0\)。
将各变量以矩阵形式表示如下：
\[
Y=\underbrace{\left[\begin{array}{c}
Y_1 \\
Y_2 \\
\ldots \\
Y_T
\end{array}\right]}_{T \times 1}, X=\underbrace{\left[\begin{array}{ccccc}
1 & X_{11} & X_{12} & \ldots & X_{1 K} \\
1 & X_{21} & X_{22} & \ldots & X_{2 K} \\
\ldots & \ldots & \ldots & \ldots & \ldots \\
1 & X_{T 1} & X_{T 2} & \ldots & X_{T K}
\end{array}\right]}_{T \times(K+1)}, \beta=\underbrace{\left[\begin{array}{c}
\beta_0 \\
\beta_1 \\
\ldots \\
\beta_K
\end{array}\right]}_{(K+1) \times 1}, u=\underbrace{\left[\begin{array}{c}
u_1 \\
u_2 \\
\ldots \\
u_T
\end{array}\right]}_{T \times 1}
\]
矩阵\(X\)中的每一行对应一个观测点，每一列对应一个解释变量。
简化为：
$$
Y=X \beta+u\label{1}\tag{1}
$$
公式\(\eqref{1}\)是真实的线性模型，其中的\(\beta\)是真实存在但未知的。我们需要构造估计量来估计\(\beta\)：
$$
Y=X \widehat{\beta}+e\tag{2}\label{2}
$$
回归系数的普通最小二乘估计量
“普通最小二乘”的含义是：使得误差项的平方和最小。普通最小二乘估计量是如下最小化问题的解：
$$
\underset{\widehat{\beta}}{\operatorname{argmin}} \sum_{t=1}^T e_{t} ^{2} = e^{\prime} e=(Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta})
$$
其中
\[
e=\underbrace{\left[\begin{array}{c}
e_1 \\
e_2 \\
\ldots \\
e_T
\end{array}\right]}_{T \times 1}
\]
解这个最小化问题的推导过程在 普通最小二乘法的矩阵形式推导 中已有介绍。这里简单回顾如下：
$$
\min (Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta})
$$
由于 \(\frac{d\left(A^{\prime} A\right)}{d A}=2 A, \frac{d(C B)}{d B}=C^{\prime}\)
$$
\begin{aligned}
\frac{d(Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta})}{d \widehat{\beta}} & =0 \\
(-X)^{\prime} 2(Y-X \widehat{\beta}) & =0 \\
X^{\prime}(Y-X \widehat{\beta}) & =0 \\
X^{\prime} Y-X^{\prime} X \widehat{\beta} & =0 \\
X^{\prime} X \widehat{\beta} & =X^{\prime} Y
\end{aligned}
$$
若 \(\left(X^{\prime} X\right)^{-1}\) 存在，则
$$
\left(X^{\prime} X\right)^{-1} X^{\prime} X \widehat{\beta}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y
$$
\[
\Rightarrow \beta^{O L S}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y
\]
一元线性回归
若只有一个解释变量，则
\[
Y=\underbrace{\left[\begin{array}{c}
Y_1 \\
Y_2 \\
\ldots \\
Y_T
\end{array}\right]}_{T \times 1}, X=\underbrace{\left[\begin{array}{cc}
1 & X_1 \\
1 & X_2 \\
\ldots & \ldots \\
1 & X_T
\end{array}\right]}_{T \times 2}, u=\underbrace{\left[\begin{array}{c}
u_1 \\
u_2 \\
\ldots \\
u_T
\end{array}\right]}_{T \times 1}, \beta=\underbrace{\left[\begin{array}{c}
\beta_0 \\
\beta_1
\end{array}\right]}_{2 \times 1}
\]
\[
\begin{aligned}
\underbrace{\left[\begin{array}{c}
\beta_0^{O L S} \\
\beta_1^{O L S}
\end{array}\right]}_{2 \times 1}& =\underbrace{\left(\left[\begin{array}{cccc}
1 & 1 & \ldots & 1 \\
X_1 & X_2 & \ldots & X_T
\end{array}\right]\left[\begin{array}{cc}
1 & X_1 \\
1 & X_2 \\
\ldots & \ldots \\
1 & X_T
\end{array}\right]\right)^{-1}}_{2 \times 2} \underbrace{\left[\begin{array}{cccc}
1 & 1 & \ldots & 1 \\
X_1 & X_2 & \ldots & X_T
\end{array}\right]\left[\begin{array}{c}
Y_1 \\
Y_2 \\
\ldots \\
Y_T
\end{array}\right]}_{2 \times 1} \\
& =\left(\left[\begin{array}{cc}
T & \sum X_t \\
\sum X_t & \sum X_t^2
\end{array}\right]\right)^{-1}\left[\begin{array}{c}
\sum Y_t \\
\sum X_t Y_t
\end{array}\right] \\
& =\frac{1}{T \sum X_t^2-\left(\sum X_t\right)^2}\left[\begin{array}{cc}
\sum X_t^2 & -\sum X_t \\
-\sum X_t & T
\end{array}\right]\left[\begin{array}{c}
\sum Y_t \\
\sum X_t Y_t
\end{array}\right] \\
& =\frac{1}{T \sum X_t^2-\left(\sum X_t\right)^2}\left[\begin{array}{c}
\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t \\
-\sum X_t \sum Y_t+T \sum X_t Y_t
\end{array}\right] \\
& =\left[\begin{array}{c}
\frac{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\
\frac{T \sum X_t Y_t-\sum X_t \sum Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2}
\end{array}\right] \\
&
\end{aligned}
\]
\(\beta_1\)的估计
\[
\begin{aligned}
\beta_1^{O L S} & =\frac{T \sum X_t Y_t-\sum X_t \sum Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\
& =\frac{\frac{1}{T^2}\left\{T \sum X_t Y_t-\sum X_t \sum Y_t\right\}}{\frac{1}{T^2}\left\{T \sum X_t^2-\left(\sum X_t\right)^2\right\}} \\
& =\frac{\frac{\sum X_t Y_t}{T}-\frac{\sum X_t}{T} \frac{\sum Y_t}{T}}{\frac{\sum X_t^2}{T}-\left(\frac{\sum X_t}{T}\right)^2} \\
& =\frac{\overline{X Y}-\overline{X} * \overline{Y}}{\overline{X^2}-\left(\overline{X}\right)^2}
\end{aligned}
\]
\(\beta_0\)的估计
\[
\begin{aligned}
\beta_0^{O L S} & =\frac{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\
& =\frac{\frac{1}{T^2}\left\{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t\right\}}{\frac{1}{T^2}\left\{T \sum X_t^2-\left(\sum X_t\right)^2\right\}} \\
& =\frac{\frac{\sum X_t^2 \sum Y_t}{T}-\frac{\sum X_t}{T} \frac{\sum X_t Y_t}{T}}{\frac{\sum X_t^2}{T}-\left(\frac{\sum X_t}{T}\right)^2} \\
& =\frac{\overline{X^2} * \overline{Y}-\overline{X} * \overline{X Y}}{\overline{X^2}-\left(\overline{X}\right)^2}
\end{aligned}
\]