What is the smallest unit of information?

I've been thinking about this off and on for years. I don't remember when first I started, but I know that this idea was inspired by the linguistic concept of morphemes, the smallest combination of sound and meaning in a language. As an example, the English word "fishing," has two morphemes, "fish," and "-ing." Then, since teaching myself programming, I've considered other types of small units of information, like bits, the smallest units of digital information. There are also genes, the smallest units of genetic information, and memes, the smallest unit of cultural replication (as per Richard Dawkins). What I want to do with this post is think about this more abstractly. The two main questions that interest me are what is information? and how can we measure it?

To put into better perspective what I mean when I say "smallest unit of information," consider the phrase "over there." Depending on the context, "over there" could referenc

Information theory as a field started in the 40's with a paper titled A Mathematical Theory of Communication by Claude E. Shannon. The first thing that stood out to me while reading this is that I was way out of my technial depth, knowing almost nothing about electrical engineering. The main thing that Shannon discusses is how to make these signals more efficent and more effective at transmitting a message with minimal noise. In this view, then, information is just the stuff that makes up a message. While the article is interesting and provides potentially interesting background, I don't think it's really relevant.

Works Cited