(3 units). Measure of information: entropy, relative entropy, mutual information, asymptotic equipartition property, entropy rates for stochastic processes; Data compression: Huffman code, arithmetic coding; Channel capacity: random coding bound, reliability function, Blahut-Arimoto algorithm, Gaussian channels, coloured Gaussian noise and "water-filling"; Rate distortion theory; Network information theory. This course is equivalent to SYSC 5506 at Carleton University. Course Component: Lecture
(3 units). Measure of information: entropy, relative entropy, mutual information, asymptotic equipartition property, entropy rates for stochastic processes; Data compression: Huffman code, arithmetic coding; Channel capacity: random coding bound, reliability function, Blahut-Arimoto algorithm, Gaussian channels, coloured Gaussian noise and "water-filling"; Rate distortion theory; Network information theory. This course is equivalent to SYSC 5506 at Carleton University. Course Component: Lecture
