The document discusses extending Shannon's information theory to incorporate semantics. It proposes a three-level model of communication including physical, technical, and semantic layers. It explores how semantics can reduce source redundancy through shared background knowledge, and defines notions of semantic noise and channel capacity. Future work is outlined on topics like lossy semantic compression, semantic mismatches, and applications to knowledge representation languages.

1. Towards a Theory of Semantic Communication Jie Bao , Prithwish Basu, Mike Dean, Craig Partridge, Ananthram Swami, Will Leland and Jim Hendler RPI, Raytheon BBN, and ARL IEEE Network Science Workshop 2011, West Point, June 23rd, 20011

6. Shannon, 1948 message message Shannon Model Signal Signal Expressed Message (e.g., commands and reports) Expressed Message From IT to SIT (Classical) Information Theory Semantic Information Theory Semantic Channel

7. A 3-level Model (adapted from Weaver) Transmitter Receiver Destination Source Physical Channel Technical message Technical Noise Intended message Expressed message Semantic Transmitter Semantic Receiver Semantic Noise Shared knowledge Local knowledge Local knowledge C: Effectiveness B: Semantic A: Technical Context, Utility, Trust etc.

8. A Semantic Communication Model Message generator World model Background Knowledge Inference Procedure Messages Sender Message interpreter World model Background Knowledge Inference Procedure Receiver W s W r K s K r I s I r {m} World M: Message Syntax Feedback (?) observations Ms Mr

10. Which message is more “surprising”? Rex is not a tyrannosaurus Rex is not a dog This slide contains animation

23. backup

25. Shannon: Information = “surpriseness” H( tyrannosaurus ) > H(dog) Captured from: http://www.wordcount.org/main.php

28. Semantic Noise and Channel Coding “ coffee machine” “ copy machine” “ Xerox ” “ Xerox” “ copy machine” p->ff ? ? 0.9 0.1 1.0 W X Y W’ Scenario developed based on reports in http://english.visitkorea.or.kr/enu/AK/AK_EN_1_6_8_5.jsp and http://blog.cleveland.com/metro/2011/03/identifying_photocopy_machine.html

29. Compressing by semantic Ambiguity Sunny Rain Light Rain Heavy Rain Sunny Light Rain Heavy Rain Status 0.5 0.25 0.25 0.5 0.2 0.2 0.1 (a) (b) H(X)=1.5 H(X’)=1.76