The GATE Grind

GATE 2024 DA – Question 22

Machine Learning · Dimensionality reduction and principal component analysis · 1 mark · Multiple choice

For any binary classification dataset, let $S_B \in \mathbb{R}^{d \times d}$ and $S_W \in \mathbb{R}^{d \times d}$ be the between-class and within-class scatter (covariance) matrices, respectively. The Fisher linear discriminant is defined by $u^* \in \mathbb{R}^d$, that maximizes $J(u) = \frac{u^TS_Bu}{u^TS_Wu}$.

If $\lambda = J(u^*)$, $S_W$ is non-singular and $S_B \neq 0$, then $(u^*, \lambda)$ must satisfy which ONE of the following equations?

  1. $S_W^{-1}S_Bu^* = \lambda u^*$
  2. $S_Wu^* = \lambda S_Bu^*$
  3. $S_BS_Wu^* = \lambda u^*$
  4. $u^{*T}u^* = \lambda^2$

Practise this question in The GATE Grind →

Show answer and explanation

Correct answer: (A) $S_W^{-1}S_Bu^* = \lambda u^*$

Explanation

Setting the gradient of the ratio $J(u)$ to zero gives $S_Bu(u^TS_Wu) = S_Wu(u^TS_Bu)$, that is $S_Bu = \lambda S_Wu$. With $S_W$ invertible, this is the eigenvalue problem $S_W^{-1}S_Bu^* = \lambda u^*$, and $\lambda$ is the maximum eigenvalue.