Mathematical information measures: entropy, relative entropy and mutual information. Assymptotic equipartition property, entropy rates of stochastic processes. Lossless source encoding theorems and source coding techniques. Channel capacity, differential entropy and the Gaussian channel. Lossy source coding rate distortion theory. Brief overview of network information theory.
Prerequisite: Graduate status, EL 6303.