GATE EC 2017 Set 1 — Question 21
Go beyond PYQs with Success TrackerAI-powered personalised practice and doubt support. Unlimited practice on eligible plans; AI usage limits apply.NAT+1 / -0EasyConditional Entropy & Mutual InfoInformation TheoryCommunications
Communications → Information Theory → Conditional Entropy & Mutual Info
Last updated
Question
Let be independent random variables. has mean and variance , while has mean and variance . The mutual information between and in bits is ________.
Correct answer
0 to 0
Solution
Mutual information quantifies the amount of information obtained about one random variable through observing the other random variable. It is defined as:
where
Similarly, .Thus, for independent random variables, the mutual information is always zero.The given means and variances are irrelevant for calculating mutual information when the variables are independent.The final answer is
where
H(X) is the entropy of and is the conditional entropy of given .For independent random variables and , the knowledge of one variable does not provide any information about the other. Therefore, the conditional entropy is equal to the entropy , and is equal to .So, .Similarly, .Thus, for independent random variables, the mutual information is always zero.The given means and variances are irrelevant for calculating mutual information when the variables are independent.The final answer is
Continue learning with Success Tracker
A step still unclear? Work through it with support
Use Success Tracker to ask about the reasoning, then try another GATE EC question to check your understanding.
AI-powered practice· Unlimited practice on eligible plans
- PYQs with solutions
- Attempt available previous-year questions, then compare your reasoning with the worked solution. Coverage varies by stream.
- Practice that adapts
- Choose a topic, work on weaker areas and bookmark questions to revisit. Your attempts feed your progress tracking.
- AI doubt support
- Ask follow-up questions about a step or concept while practising, instead of stopping at the final answer.
Unlimited practice is available on eligible plans. Free practice and AI usage have limits; check the current plan allowances before choosing.
This page stays readable without an account. AI responses can be wrong; check them against the solution and source material.
More questions on Information Theory
2026 Set 1 Q22Consider a discrete memoryless source with an alphabet of four source symbols. is a…2026 Set 1 Q32A wireless digital transmission scheme is using -QAM over an additive white Gaussian noise…2026 Set 1 Q44A QPSK modulated signal from an additive white Gaussian noise (AWGN) channel is received with an…2026 Set 1 Q55Let the relevant bandwidth () of a digital communication system be and…2026 Set 1 Q57Consider a real, narrowband signal where the maximum…