GATE 2022 EC – Question 29
Let $H(X)$ denote the entropy of a discrete random variable $X$ taking $K$ possible distinct real values. Which of the following statements is/are necessarily true?
Practise this question in The GATE Grind →
Show answer and explanation
Correct answer: (A) $H(X)\le\log_2K$ bits; (B) $H(X)\le H(2X)$; (D) $H(X)\le H(2^X)$
Explanation
Entropy is maximised by the uniform distribution, so $H(X)\le\log_2K$. The maps $x\to2x$ and $x\to2^x$ are one-to-one, so they preserve the probabilities and $H(2X)=H(2^X)=H(X)$. The map $x\to x^2$ can merge $\pm a$ and can only lose information, so $H(X^2)\le H(X)$ and C is not necessarily true.