Efficient Coding

The term coding, as applied to electrical communication, has several meanings. It means the representation of letters as sequences of dots and dashes. It means the representation of signal sample amplitudes as groups of pulses having two or more possible amplitudes as in pulse code modulation. Lately, it has also come to be the generic term for any process by which a message or message wave is converted into a signal suitable for a given channel. In this usage single-sideband modulation, frequency modulation and pulse code modulation are examples of encoding procedures, while microphones, teletypewriters and television cameras are examples of encoding devices. This is a nice concept, but it is useful to distinguish between two classes of encoding processes and devices: those which make no use of the statistical properties of the signal, and those which do. In the first class, the encoding operation consists simply of a one-to-one conversion of the message into a new physical variable, as a microphone converts sound pressure into a proportional voltage or current, or of the one-to-one remapping of the message into a new representation without regard to probabilities, as by ordinary amplitude, frequency or pulse code modula724