Mathematical Psychology
About

Entropy in Perception

Information-theoretic approaches to perception quantify the information content of stimuli, the channel capacity of perceptual systems, and the uncertainty reduction achieved by sensory processing.

T(S;R) = H(S) − H(S|R)

Wendell Garner's (1962) Uncertainty and Structure as Psychological Concepts established the systematic application of information theory to perception. Garner proposed that perceptual experience is fundamentally about the reduction of uncertainty: a stimulus is informative to the degree that it narrows the set of possible states of the world. This framework provided a rigorous quantitative language for analyzing stimulus structure, perceptual grouping, dimensional interactions, and the limits of sensory information processing.

Uncertainty Analysis

Information Transmission in Perception H(S) = stimulus entropy = −Σ p(sᵢ) · log₂ p(sᵢ)
H(R) = response entropy = −Σ p(rⱼ) · log₂ p(rⱼ)
T(S;R) = transmitted information = H(S) + H(R) − H(S,R)

Channel capacity: C = max T(S;R) ≈ 2–3 bits (unidimensional)

In Garner's framework, the experimenter controls stimulus entropy H(S) by manipulating the number and probabilities of stimuli. The observer's responses have their own entropy H(R), and the information transmitted T(S;R) — the mutual information between stimulus and response — measures how much of the stimulus uncertainty the observer has resolved. The difference H(S) − T(S;R) = H(S|R) is the equivocation, or the uncertainty about the stimulus that remains after observing the response.

Stimulus Structure and Goodness

Garner extended information-theoretic analysis to the perception of pattern structure. He proposed that the "goodness" of a pattern — how regular, simple, or well-organized it appears — is inversely related to the entropy of the set of patterns that can be generated by applying symmetry transformations to it. A square has low entropy (few distinct rotations and reflections produce new patterns) and appears "good"; an irregular polygon has high entropy (many transformations produce distinct patterns) and appears complex.

Integral versus Separable Dimensions

Garner and Felfoldy (1970) used information-theoretic measures to distinguish integral and separable stimulus dimensions. Integral dimensions (like the hue and saturation of a color) interact in perception: they produce redundancy gains when correlated and interference when varied orthogonally. Separable dimensions (like the color and shape of an object) are processed independently: performance on one dimension is unaffected by variation on the other. This classification has implications for perceptual channel capacity — integral dimensions effectively share a common channel, while separable dimensions may use parallel channels.

Perceptual Channel Capacity

The channel capacity of the human perceptual system depends on both the stimulus dimension and the judgment type. For absolute identification of unidimensional stimuli, capacity is remarkably consistent at about 2–3 bits across modalities and dimensions — pitch, loudness, line length, brightness, and saltiness all yield similar limits. For relative judgments (comparing two simultaneously presented stimuli), capacity is much higher, reflecting the observation that the bottleneck lies in the labeling process rather than in sensory resolution per se.

Garner's information-theoretic approach remains influential in modern perception research. Contemporary work on perceptual decision making, visual search, and ensemble perception continues to use entropy and mutual information to characterize how the visual system extracts and represents statistical regularities in the environment.

Related Topics

References

  1. Garner, W. R. (1962). Uncertainty and Structure as Psychological Concepts. Wiley.
  2. Garner, W. R., & Felfoldy, G. L. (1970). Integrality of stimulus dimensions in various types of information processing. Cognitive Psychology, 1(3), 225–241. doi:10.1016/0010-0285(70)90016-2
  3. Garner, W. R., & Hake, H. W. (1951). The amount of information in absolute judgments. Psychological Review, 58(6), 446–459. doi:10.1037/h0054482
  4. Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61(3), 183–193. doi:10.1037/h0054663
  5. Norwich, K. H. (1993). Information, Sensation, and Perception. Academic Press.

External Links