The efficient coding hypothesis, proposed by Horace Barlow in 1961, states that the goal of early sensory processing is to recode incoming signals into a format that transmits as much information as possible given the constraints imposed by neural resources. In its original formulation, Barlow proposed "redundancy reduction": sensory neurons should remove statistical redundancy in natural stimuli, producing a factorial code in which neural responses are statistically independent. This principle connects neural architecture to Shannon's source coding theorem and has driven decades of research on neural coding.
Redundancy Reduction and Efficient Coding
Subject to: ⟨C(R)⟩ ≤ C_max (metabolic constraint)
For a noiseless system: maximize H(R)
→ histogram equalization: P(r) = uniform
→ neural response function matches the cumulative distribution of stimuli
In the simplest case of a single noiseless neuron with a limited output range, maximizing information transmission requires histogram equalization: the neuron's response function should be chosen so that its outputs are uniformly distributed. This means the response function should equal the cumulative distribution function of the input stimulus ensemble. Laughlin (1981) confirmed this prediction by showing that the contrast-response function of large monopolar cells in the fly visual system closely matches the cumulative distribution of contrasts in natural scenes.
Extensions to Neural Populations
For populations of neurons, efficient coding predicts that neural responses should be decorrelated — each neuron should carry independent information. Atick and Redlich (1992) showed that the receptive field properties of retinal ganglion cells can be explained as whitening filters that decorrelate the spatially correlated statistics of natural images. In low-noise conditions, efficient coding predicts whitening (complete decorrelation); in high-noise conditions, it predicts smoothing (preserving low spatial frequencies where signal-to-noise ratio is high). This noise-dependent tradeoff accounts for changes in retinal processing across different light levels.
Olshausen and Field (1996) extended efficient coding from decorrelation to statistical independence, showing that maximizing the sparseness and independence of neural responses to natural images produces receptive fields resembling the oriented, bandpass filters observed in primary visual cortex (V1). This result — that the receptive field properties of V1 simple cells emerge as the statistically optimal representation of natural image statistics — was a landmark confirmation of the efficient coding hypothesis and connected it to independent component analysis (ICA) and sparse coding algorithms.
Metabolic Constraints and Neural Resource Allocation
Modern formulations of efficient coding explicitly incorporate metabolic costs. Each spike costs energy (approximately 10⁸ ATP molecules), and the brain, consuming roughly 20% of the body's metabolic budget, operates under stringent energy constraints. Levy and Baxter (1996) showed that the observed sparseness of neural activity — most neurons fire at low rates most of the time — is consistent with maximizing information transmission per unit energy cost. The optimal firing rate distribution given a metabolic constraint is exponential, with most neurons silent and a few highly active, matching the observed lognormal distribution of cortical firing rates.
The efficient coding hypothesis continues to generate predictions about neural coding across sensory systems. In audition, the tonotopic organization of the cochlea and the filter properties of auditory nerve fibers can be explained as efficient codes for the statistics of natural sounds. In olfaction, the high dimensionality of odorant space and the distributed coding observed in olfactory receptor neurons are consistent with efficient representation of the complex statistics of natural odor environments. The framework provides a principled link between ecological statistics, neural architecture, and information-theoretic optimality.