The GATE Grind

GATE 2020 EC – Question 32

Communications · Information Theory · 1 mark · Numerical answer

A binary random variable $X$ takes the value $+2$ or $-2$. The probability $P(X=+2)=\alpha$. The value of $\alpha$ (rounded off to one decimal place), for which the entropy of $X$ is maximum, is ________.

Practise this question in The GATE Grind →

Show answer and explanation

Correct answer: 0.49 to 0.51

Explanation

The entropy of a binary variable is $H=-\alpha\log\alpha-(1-\alpha)\log(1-\alpha)$. It measures uncertainty and is largest when the two outcomes are equally likely, so $\alpha=0.5$. (The values $\pm2$ are irrelevant; only the probabilities matter.)

More Information Theory questions

All Information Theory questions →