This site uses only proprietary and third party technical cookies. By continuing to browse the site you are agreeing to our use of cookies. I agree I want to find out more
Browse the Department site:
Browse the Teaching site:

Programme of Course "Teoria dell'Informazione"



Type of course unit:

Bachelor Degree in Computer Science curriculum General: Elective
Master Degree in Computer Science curriculum GSEEM: Elective
Master Degree in Computer Science curriculum General: Elective

Level of course unit:

Undergraduate Degrees
Postgraduate Degrees


2nd semester

Number of credits:

Bachelor Degree in Computer Science: 6 (workload 150 hours)
Master Degree in Computer Science: 6 (workload 150 hours)


Filippo Mignosi (FilippodotMignosiatunivaqdotit)

1. Course Objectives

Knowledge of the fondamental concepts of Information Theory and ability to manipulate them  formally. Deep understanding of common-sense concepts like "information", "representation", "learning", "model". Ability to translate intuitive solutions constructed with such concepts into concrete applications in different technological areas

2. Course Contents and learning outcomes (Dublin Descriptors)

Topics of the course include:

  • Information and entropy: joint, conditional, and mutual.
  • Relative measures, AEP and its consequences.
  • Representation and codes: uniform, variable length, and adaptive.  Kraft-McMillan Inequality and its consequences.
  • Souce Coding and Compression algorithms (Shannon, Arithmetic, Huffmann, Ziv and Lempel coding, integer encoding, gamma and omega Elias coding)
  • Basic concepts of channel coding, channel capacity. 
  • Basic concepts of modern Cryptography.

On successful completion of this course, the student should

  • understand and explain fundamental concepts such as entropy, mutual information, capacity,  compression, coding theorem,  coding theory, coding and Cryptography; compute entropy and mutual information of random variables; formulate and prove The main theorems treated such as: i) AEP and its consequences,  ii) Optimality of Huffmann coding, and of arithmetical coding,  iii) the entropy is a lower bound for the expected length of a u.d block code,  iv) if P is different from NP then there exists no perfectly secret encryption scheme  with key shorter than the message.

  • understand and apply fundamental concepts in information theory such as probability,  entropy, information content and their inter-relationships, AEP, data compression; implement and analyze basic coding and compression algorithms;

  • be able to evaluate the aspects of information theory that can be applied in the real world. The student  should also be able to link the theoretical aspect of the discipline to the practical ones (such as  data compression)
  • explain how information theory and coding contributes to modern communications technology; be able to describe the main results of information theory to other non-specialist people  in the scientific community.

  • Be able to read and understand books and papers concerning the arguments treated in the course. Solve advanced problems in the area.

3. Course Prerequisites

Basic probability and discrete mathematics. Ability to develop software applications.

4. Teaching methods and language

Italian and english


Reference textbooks

  • Arora, Barak, Computational Complexity: A Modern Approach, . Cambridge University press . 2009.
  • Cover e Thomas, Elements of Information Theory. 2006.

5. Assessment Methods

Written and oral.

Course information last updated on: 08 giugno 2017, 10:38