GATE 2016 EC – Question 58
Consider a discrete memoryless source with alphabet $S = \{s_0, s_1, s_2, s_3, s_4, \ldots\}$ and respective probabilities of occurrence $P = \left\{\frac{1}{2}, \frac{1}{4}, \frac{1}{8}, \frac{1}{16}, \frac{1}{32}, \ldots\right\}$. The entropy of the source (in bits) is ________
Practise this question in The GATE Grind →
Show answer and explanation
Correct answer: 1.8 to 2.2
Explanation
Information content of $s_k$ with probability $2^{-(k+1)}$ is $(k+1)$ bits, so
$H=\sum_{k=0}^{\infty}2^{-(k+1)}(k+1)=\sum_{n=1}^{\infty}\dfrac n{2^n}$.
This standard series equals 2 (use $\sum nx^n=\dfrac x{(1-x)^2}$ with $x=\tfrac12$: $\dfrac{1/2}{1/4}=2$). So $H=2$ bits.