22. Quadratic Expansion of the GMM Objective Function
Article: "[[33+. On asymptotic size distortions in the random coefficients logit model]]"
Part: Eq (2)
Settings
- Assuming there are $l$ moment conditions and $k$ parameters, let the second-order Taylor expansion of the GMM objective function ($Q_T(\theta) = \frac{1}{2} G_T(\theta)'\mathcal{W}G_T(\theta)$) be:$$Q_T(\theta) = Q_T(\theta_T) + G_T(\theta_T)'\mathcal{W}G_\theta \delta + \frac{1}{2}\delta'G_\theta'\mathcal{W}G_\theta\delta + R_T(\theta) \tag{1}$$where the parameter difference is $\delta_{(k \times 1)} = \theta_{(k \times 1)} - \theta_{T(k \times 1)}$. Here, $G_T(\cdot)$ is an $l \times 1$ sample moment vector, $\mathcal{W}$ is an $l \times l$ weight matrix, and $G_\theta$ is the $l \times k$ Jacobian matrix. Note that for a finite sample size $T$, both $Q_T(\theta)$ and $Q_T(\theta_T)$ are scalar random variables.
- Define the symmetric information matrix $\mathcal{J}_{(k \times k)}$ and the scaled unconstrained minimizer $\mathcal{Z}_{T(k \times 1)}$ (For the underlying intuition, see # Derivation Intuition): $$\mathcal{J} = G_\theta'\mathcal{W}G_\theta \tag{2}$$ $$\mathcal{Z}_T = -\mathcal{J}^{-1}G_\theta'\mathcal{W}\sqrt{T}G_T(\theta_T) \tag{3}$$
- Objective: Express $Q_T(\theta)$ as a shifted quadratic form using the defined scalar function $q_T(\lambda) = (\lambda - \mathcal{Z}_T)'\mathcal{J}(\lambda - \mathcal{Z}_T)$, where $\lambda$ is a $k \times 1$ vector.
Proof
- Isolate the first-order derivative term from $\mathcal{Z}_T$:
Multiply $\mathcal{Z}_T$ by $-\frac{1}{\sqrt{T}}\mathcal{J}$ and take the transpose (noting $\mathcal{J} = \mathcal{J}'$ and $\mathcal{W} = \mathcal{W}'$): $$\Big[- \frac{1}{\sqrt{T}}\mathcal{Z}_T'\mathcal{J}\Big]_{1 \times k} = G_T(\theta_T)'\mathcal{W}G_\theta \tag{4}$$
- Substitu te $\mathcal{J}$ and the derived first-order term into the original expansion: $$Q_T(\theta) = Q_T(\theta_T) - \frac{1}{\sqrt{T}} \mathcal{Z}_T' \mathcal{J} \delta + \frac{1}{2} \delta' \mathcal{J} \delta + R_T(\theta) \tag{5}$$
- Expand the target quadratic function $q_T(\lambda)$ evaluated at the local parameter $\lambda = \sqrt{T}\delta$: $$\begin{aligned} q_T(\sqrt{T}\delta) &= (\sqrt{T}\delta - \mathcal{Z}_T)'\mathcal{J}(\sqrt{T}\delta - \mathcal{Z}_T) \\ &= T\delta'\mathcal{J}\delta - \sqrt{T}\delta'\mathcal{J}\mathcal{Z}_T - \sqrt{T}\mathcal{Z}_T'\mathcal{J}\delta + \mathcal{Z}_T'\mathcal{J}\mathcal{Z}_T \end{aligned} \tag{6}$$
Since $\delta'_{(1 \times k)}\mathcal{J}_{(k \times k)}\mathcal{Z}_{T(k \times 1)}$ is a $1 \times 1$ scalar, it equals its transpose $\mathcal{Z}_T'\mathcal{J}\delta$. Thus: $$q_T(\sqrt{T}\delta) = T\delta'\mathcal{J}\delta - 2\sqrt{T}\mathcal{Z}_T'\mathcal{J}\delta + \mathcal{Z}_T'\mathcal{J}\mathcal{Z}_T \tag{7}$$
- Scale $q_T(\sqrt{T}\delta)$ by $\frac{1}{2T}$ to match the coefficients in Step 2: $$\frac{1}{2T}q_T(\sqrt{T}\delta) = \underbrace{\frac{1}{2}\delta'\mathcal{J}\delta - \frac{1}{\sqrt{T}}\mathcal{Z}_T'\mathcal{J}\delta}_{\text{Rearrange}} + \frac{1}{2T}\mathcal{Z}_T'\mathcal{J}\mathcal{Z}_T \tag{8}$$
- Rearrange to isolate the core first and second-order terms of $Q_T(\theta)$: $$- \frac{1}{\sqrt{T}}\mathcal{Z}_T'\mathcal{J}\delta + \frac{1}{2}\delta'\mathcal{J}\delta = \frac{1}{2T}q_T(\sqrt{T}\delta) - \frac{1}{2T}\mathcal{Z}_T'\mathcal{J}\mathcal{Z}_T \tag{9}$$
Substitute this equivalence back into the equation derived in Step 2.
Conclusion
The GMM objective function can be exactly rewritten as a localized quadratic form: $$Q_T(\theta) = Q_T(\theta_T) - \frac{1}{2T}\mathcal{Z}_T'\mathcal{J}\mathcal{Z}_T + \frac{1}{2T}q_T(\sqrt{T}(\theta - \theta_T)) + R_T(\theta) \tag{10}$$
Q.E.D.
Derivation Intuition
The construction of $q_T(\lambda)$ and $\mathcal{Z}_T$ is not arbitrary. It is the high-dimensional matrix equivalent of finding the minimum of a parabola and completing the square (配方法).
1. Finding the Unconstrained Minimum
Ignoring the constant $Q_T(\theta_T)$ and the remainder $R_T(\theta)$, the core objective function with respect to the parameter difference $\delta = \theta - \theta_T$ is: $$f(\delta) = G_T'\mathcal{W}G_\theta \delta + \frac{1}{2}\delta'\mathcal{J}\delta \tag{11}$$
To find the theoretical "bottom of the bowl" (assuming no boundary constraints), we take the first-order condition with respect to $\delta$: $$\Big[\frac{\partial f(\delta)}{\partial \delta} \Big]_{k \times 1} = G_\theta'\mathcal{W}G_T + \mathcal{J}\delta = 0 \tag{12}$$
Solving for the unconstrained optimal $\delta^*$: $$\delta^* = -\mathcal{J}^{-1}G_\theta'\mathcal{W}G_T \tag{13}$$
2. Definition of $\mathcal{Z}_T$
In large samples, the parameter difference $\delta^*$ shrinks to $0$. To prevent it from collapsing, we amplify it by scaling by $\sqrt{T}$. This scaled unconstrained minimum is precisely the definition of $\mathcal{Z}_T$: $$\sqrt{T}\delta^* = -\mathcal{J}^{-1}G_\theta'\mathcal{W}(\sqrt{T}G_T) \equiv \mathcal{Z}_T \tag{14}$$
3. Completing the Square
With the unconstrained minimum $\mathcal{Z}_T$ representing the "bullseye" and the Hessian $\mathcal{J}$ representing the "steepness of the bowl", the quadratic form for any localized parameter $\lambda = \sqrt{T}\delta$ is beautifully constructed as the distance to the minimum: $$q_T(\lambda) = (\lambda - \mathcal{Z}_T)'\mathcal{J}(\lambda - \mathcal{Z}_T) \tag{15}$$
[!TIP]
This is exactly the matrix version of completing the square in middle school math: $y = ax^2 + bx \implies y = a(x - x^*)^2 + C$, where the extremum $x^* = -\frac{b}{2a}$ corresponds to $\mathcal{Z}_T$ and the curvature $a$ corresponds to the information matrix $\mathcal{J}$.

