GATE EC 2016 Set 2 — Question 31
Go beyond PYQs with Success TrackerAI-powered personalised practice and doubt support. Unlimited practice on eligible plans; AI usage limits apply.NAT+1 / -0EasyInformation Measure & EntropyInformation TheoryCommunicationsSource & Channel Coding
Communications → Information Theory → Information Measure & Entropy
Last updated
Question
A discrete memoryless source has an alphabet with corresponding probabilities . The minimum required average codeword length in bits to represent this source for error-free reconstruction is ________
Correct answer
1.74 to 1.76
Solution
For a discrete memoryless source, Shannon's Source Coding Theorem states that the minimum average codeword length required for error-free reconstruction is bounded by the entropy
The entropy
bits/symbolGiven the probabilities:
Substituting these values into the entropy formula:
bits/symbolSince all probabilities are of the form (where is an integer), a prefix-free code such as the Huffman code will achieve an average length exactly equal to the entropy. Thus, the minimum required average codeword length is 1.75 bits.
H(X) of the source:The entropy
H(X) is calculated as:bits/symbolGiven the probabilities:
Substituting these values into the entropy formula:
bits/symbolSince all probabilities are of the form (where is an integer), a prefix-free code such as the Huffman code will achieve an average length exactly equal to the entropy. Thus, the minimum required average codeword length is 1.75 bits.
Continue learning with Success Tracker
A step still unclear? Work through it with support
Use Success Tracker to ask about the reasoning, then try another GATE EC question to check your understanding.
AI-powered practice· Unlimited practice on eligible plans
- PYQs with solutions
- Attempt available previous-year questions, then compare your reasoning with the worked solution. Coverage varies by stream.
- Practice that adapts
- Choose a topic, work on weaker areas and bookmark questions to revisit. Your attempts feed your progress tracking.
- AI doubt support
- Ask follow-up questions about a step or concept while practising, instead of stopping at the final answer.
Unlimited practice is available on eligible plans. Free practice and AI usage have limits; check the current plan allowances before choosing.
This page stays readable without an account. AI responses can be wrong; check them against the solution and source material.
More questions on Information Theory
2026 Set 1 Q22Consider a discrete memoryless source with an alphabet of four source symbols. is a…2026 Set 1 Q32A wireless digital transmission scheme is using -QAM over an additive white Gaussian noise…2026 Set 1 Q44A QPSK modulated signal from an additive white Gaussian noise (AWGN) channel is received with an…2026 Set 1 Q55Let the relevant bandwidth () of a digital communication system be and…2026 Set 1 Q57Consider a real, narrowband signal where the maximum…