Information Theory, Speech and Writing

Claude Shannon

Information Theory provides broadly useful concepts; it describes the physical limits on recording or transmitting data. Broadly speaking, information is change. It's present on the microscopic scale in the spin of electrons, the orbital states of electrons, and chemical reactions. On the large scale, it's present in the tides, the rising and setting of the sun, gravitational fields, and the like.

The concepts can also be applied to the problem of transferring thoughts from one human mind to another. The rate of data transfer through speech or writing is small compared to the rate of data transfer through visual media, like a video. However, writing or speech does not rely solely on the information that's present in the words. From an information theory point of view, speech or writing is like a compressed form of data transmission. It relies on information that's already shared between the speaker and the listener, or the writer and the reader.

Part of that shared information is cultural, taught since childhood, or learned through study. Part of it is inborn, recorded at the level of organs, cells, and DNA of our ancestors, and even in the inert matter that led to our most ancient single cell ancestors. That information comes from the universe. The part of that shared information that's encoded from the universe doesn't necessarily translate easily to speech, because it predates words and speech by millions and billions of years.

One of the fundamental problems of communicating ideas, then, is the shared experience of the author and the audience might not overlap, so, in information theory terms, the message is transmitted with lossy compression, and only an imperfect version is received.