GATE 2026 PH – Question 53
Consider the distribution of outcomes generated by N (N≫1) independent throws of (i) a coin or (ii) a six-sided dice. A coin (dice) is unbiased if both (all) its sides have equal probability to show up in a throw; it is biased otherwise. For the cases (i) and (ii) above, which of the following statements is/are true?
Practise this question in The GATE Grind →
Show answer and explanation
Correct answer: (A) The entropy of an unbiased coin is smaller than that of an unbiased dice.; (C) The entropy of a biased dice is smaller than that of an unbiased dice.
Explanation
The **Shannon (Gibbs) entropy** of a distribution is $S=-k\sum_ip_i\ln p_i$. For $n$ equally likely outcomes, the entropy is at its maximum, $S_{max}=k\ln n$.
- **A. The entropy of an unbiased coin is smaller than that of an unbiased die.** Coin: $k\ln2$. Die: $k\ln6$. Since $\ln2<\ln6$, true. ✓
- **B. The entropy of an unbiased coin is greater than that of an unbiased die.** False, for the same reason.
- **C. The entropy of a biased die is smaller than that of an unbiased die.** True: for a fixed number of outcomes, any departure from equal probabilities lowers the entropy. ✓
- **D. The entropy of a biased coin is greater than that of an unbiased coin.** False: the unbiased coin has the maximum entropy.
Answer **A and C**.