Empirical theory of perception

An empirical theory of perception is a type of explanation for how percepts form. These theories hold that sensory systems incorporate information about the statistical properties of the natural world into their design and relate incoming stimuli to this information, rather than analyzing sensory stimulation into its components or features.

Empirical accounts of vision

Visual perception is initiated when objects in the world reflect light rays towards the eye. Most empirical theories of visual perception begin with the observation that stimulation of the retina is fundamentally ambiguous. In empirical accounts, the most commonly proposed mechanism for circumventing this ambiguity is "unconscious inference," a term that dates back to Hermann von Helmholtz.

According to Hatfield, Ibn al-Haytham was the first to propose that higher-level cognitive processes ("judgments") could supplement sense perception to lead to an accurate perception of distance, suggesting that these "judgments" are formally equivalent to syllogisms.[1] René Descartes extended and refined this account. George Berkeley departed from this tradition, putting forth the new idea that sensory systems, rather than performing logical operations on stimuli to reach veridical conclusions (i.e. these light rays come with certain orientations relative to each other, therefore their source is at a certain distance), make associations. For instance, if certain co-occurring sensory attributes are usually present when an object is at a given distance, an observer would see an object with those attributes as being at that distance. For Helmholtz, Berkeleyan associations form the premises for inductive "judgements," in al-Haytham's sense of the term. Helmholtz was one of the first thinkers on the subject to augment his reasoning with detailed knowledge of the anatomy of sensory mechanisms. Helmholtz also collected a lot of knowledge from physiology, experimental psychology, anatomy, and pathology to expand his idea about the anatomy of sensory mechanisms.[2]

In current works Helmholtz's use of the term is construed as referring to some mechanism that augments sense impressions with acquired knowledge or through application of heuristics. In general, contemporary empirical theories of perception seek to describe and/or explain the physiological underpinnings of this "unconscious inference," particularly in terms of how sensory systems acquire information about general statistical features of their environments (see natural scene statistics) and apply this information to sensory data in order to shape perception. A recurring theme in these theories is that stimulus ambiguity is rectified by a priori knowledge about the natural world.

Wholly empirical approach to visual perception

The wholly empirical approach to perception, developed by Dale Purves and his colleagues, asserts that percepts are determined solely by evolutionary and individual experience with sensory impressions and the objects from which they derive. The success or failure of behavior in response to these sensory impressions tends to increase the prevalence of neural structures that support some ways of interpreting sensory input while decreasing the prevalence of neural structures that support other ways of interpreting sensory input.[3]

This strategy determines qualities of perception in all visual domains and sensory modalities. Accumulating evidence suggests that the perception of color,[4][5] contrast,[6] distance,[7] size,[8] length, line orientation and angles,[9] and motion,[10][11] as well as pitch and consonance in music,[12][13][14] may be determined by empirically derived associations between the sensory patterns humans have always experienced and the relative success of behavior in response to those patterns.

Much to the advantage of the observer, percepts co-vary with the efficacy of past actions in response to visual stimuli, and thus only coincidentally with the measured properties of the stimulus or the underlying objects.

— Dale Purves, Purves Lab website[15]

The wholly empirical strategy

The wholly empirical theory of perception differs from other empirical theories by recognizing the severity of the inverse problem in optics. As an example, imagine that three hoses are used to fill a bucket with water. If how much water each hose has contributed is known, it is straightforward to calculate how much water is in the bucket. These kinds of problems are known as “forward” problems, which are easy to solve. But if all that is known is the amount of water in the bucket instead, it is impossible to figure out with just this information how much water came from each hose; it is impossible to work “backwards” from the bucket to the hoses. This is an example of an inverse problem. Solutions to these problems are rarely possible, although they can sometimes be approximated by imposing assumption-based constraints on the “solution space”.

Navigating the world on the basis of sensory stimulation alone represents an inverse problem in the realm of biology. When light reflected from a linear object falls on the retina, the object in 3-D space is transformed into a two-dimensional line. A distant line can form the same image on the retina as a shorter but close line. It is impossible to work backwards to know the real distance, length, and orientation of the source of the projected line. Despite this fact, observers usually manage to behave effectively in response to sensory stimulation.

The inverse optics problem presents a quandary for traditional approaches to perception. Advocates of feature detection (known today as neural filtering) propose that the visual system performs logical computations on retinal inputs to determine higher-level aspects of a perceptual scene such as contrast, contour, shape and color percepts. Given the inverse problem, it is hard to imagine how these computations would be useful as they would have little or nothing to do with properties of the real world. Empirical approaches to perception propose that the only way for organisms to successfully overcome the inverse problem is to exploit their long and varied past experience with the real world.

The wholly empirical approach states that this experience is the sole determinant of perceptual qualities. It asserts that the reason observers see an object as dark or light is that in both the individual's past and the past of the species it was advantageous to see it in that manner.

Color

Color vision is dependent on activation of three cone cell types in the human retina, each of which is primarily responsive to a different spectrum of light frequencies. While these retinal mechanisms enable subsequent color processing, their properties alone cannot account for the full range of color perception phenomena. In part this is because illuminance (the amount of light shining on an object), reflectance (the amount of light an object is predisposed to reflect), and transmittance (the extent to which the light medium distorts the light as it travels) are conflated in the retinal image. This is problematic because, if color vision is to be useful, it must somehow guide behavior in line with these properties. Even so, the visual system only has access to retinal input, which does not distinguish the relative contributions of each of these factors to the final light spectra that stimulate the retina.

According to the empirical framework, the visual system solves this problem by drawing on species and individual experience with retinal images that have signified different combinations of illuminance, reflectance, and transmittance in the past. Only those associations that led to appropriate behavior were retained through evolution and development, leading to a repertoire of neural associations and predispositions that ground color perception in the world.

Simultaneous color contrast. The smaller squares are identical in color but look different depending on context.

One way to test this idea is to see if the frequency of co-occurrence of light spectra predicts simultaneous color contrast effects. Fuhui Long and Purves[16] showed that by sampling thousands of natural images, analysis of associations between target colors and the colors of their surrounds could explain perceptual effects like those seen on the right. Rather than explaining the diverging color percepts as unfortunate byproducts of a normally veridical color perception mechanism, the different colors humans see are simply the byproducts of the species' and the individual's exposure to the distribution of color spectra in the world.

Brightness

The checker shadow illusion. Squares A and B are identical in luminance returns but look differently bright depending on context.

’’Brightness’’ refers to a subjective sense that the object considered is emitting light. Whereas the perceptual correlates of color are the frequencies of light that compose the light spectrum, the perceptual correlate of brightness is luminance (the intensity of light emitted by an object). While it may seem obvious that the sensation of brightness is straightforwardly related to the amount or intensity of light coming to the eyes, researchers studying perception have long known that brightness is not caused solely by the luminance incident on the retina.[17] A common example is simultaneous brightness contrast (shown to the right), in which the two identical target diamonds appear to have different brightnesses.

In the empirical account, the same general framework used to rationalize simultaneous color contrast applies to simultaneous brightness contrast. The visual system associates luminance values and their given contexts with the success or failure of ensuing behavior, leading to percepts that often (but only incidentally) reflect properties of objects rather than their associated images.

The checker shadow illusion strongly supports this view of how brightness perception works. Although other frameworks have either no explanation for this effect or explanations that are highly inconsistent with their explanations for similar effects, the empirical framework supports that the perceived brightness differences are due to empirical associations between the targets and their respective contexts. In this case, because the “lighter” targets would typically have been shadowed, humans perceive them in a way that is consistent with their having a higher reflectance despite their presumably low levels of illuminance. This approach is considerably different from computational “context”-driven approaches, as the target/context relationships are contingent and world-based, and therefore cannot be generalized to other cases in any meaningful way.

Line length

Perception of line length is confounded by another optical inverse problem: the further away a line is from the observer, the smaller the projected line will be on the retina. Different orientations of a line relative to the observer may obscure true line length as well. Straight lines are erroneously reported as longer or shorter as a function of their angular orientation,[18] as demonstrated by the vertical–horizontal illusion. While no generally accepted explanations of this phenomenon have been offered previously, the empirical approach has had some success in explaining the effect as a function of the distribution of lines in natural scenes.[19] Catherine Howe and Purves (2002) analyzed natural scene photographs to find projected lines that corresponded to straight line sources. They found that the ratios of the actual length of the lines to the projected lines on the retina, when classified by their respective orientations on the retina, almost perfectly matched subjective estimation of line length as a function of angle relative to the observer. For example, horizontal lines on the retinal image typically have turned out to issue from relatively short physical sources, while lines at about 60 degrees relative to the observer typically have signified longer physical sources, which explains why individuals tend to see a line at 60 degrees as longer than a horizontal line. While there is no way for the visual system to know this a priori, the fact that it seems to take this knowledge for granted in its construction of length estimation percepts strongly supports the wholly empirical view of perception.

Motion

Perception of motion is also confounded by an inverse problem: movement in three-dimensional space does not map perfectly onto movement on the retinal plane. A distant object moving at a given speed will translate more slowly on the retina than a nearby object moving at the same speed. As mentioned previously, size, distance, and orientation are also ambiguous given only the retinal image. As with other aspects of perception, empirical theorists propose that this problem is solved by trial-and-error experience with moving stimuli, their associated retinal images, and the consequences of behavior. One way to test this hypothesis is to determine if it can explain the flash lag illusion, a visual effect in which a flash superimposed on a moving bar is falsely seen to lag behind the bar. The task for empirical theorists is to explain why individuals perceive the flash in this way and why the perceived lag increases with the speed of the moving bar. To investigate this question, William Wojtach and his team simulated a three-dimensional environment full of moving virtual particles.[20] They modeled the transformation from three dimensions to the two-dimensional image plane and tallied up the frequency of occurrence of particle speeds, particle distances, image speeds, and image distances, where image means the path projected across the computer-modeled “retina”. The probability distributions they obtained predicted the magnitude of the bar-flash disparity well. The authors concluded that the flash-lag effect was a signature of the way brains evolve and develop to behave appropriately in response to moving retinal images.

References

1. Hatfield, Gary. Perception as Unconscious Inference. Accessed from http://repository.upenn.edu/ircs_reports/9/ .
2. Finger, Stanley; Wade, Nicholas J. (September 2002). "The Neuroscience of Helmholtz and the Theories of Johannes Müller". Journal of the History of the Neurosciences. 11 (3): 234–254. doi:10.1076/jhin.11.3.234.10392. ISSN 0964-704X.
3. Purves, Dale; Monson, Brian B.; Sundararajan, Janani; Wojtach, William T. (2014-04-01). "How biological vision succeeds in the physical world". Proceedings of the National Academy of Sciences. 111 (13): 4750–4755. doi:10.1073/pnas.1311309111. ISSN 0027-8424. PMC 3977276. PMID 24639506.
4. Perceiving Colour. Lotto RB, Purves D. Review of Progress in Coloration 34:12-25. (2004)
5. Natural scene statistics as the universal basis for color context effects. Long F, Purves D. Proceedings of the National Academy of the Sciences 100(25): 15190-15193. (2003).
6. An empirical explanation of the Chubb illusion. Lotto RB, Purves D. Journal of Cognitive Neuroscience 13(5): 547-555. (2001)
7. A statistical explanation of visual space. Yang Z, Purves D. Nature Neuroscience 6:632:640 (2003).
8. Size contrast and assimilation explained by the statistics of scene geometry. Howe CQ, Purves D. Journal of Cognitive Neuroscience 16(1): 90-102. (2004).
9. Natural scene geometry predict the perception of angles and line orientation. Howe CQ, Purves D. Proceedings of the National Academy of the Sciences 102(4): 1228-1233. (2005).
10. An empirical explanation of aperture effects. Sung K., Wojtach W.T., Purves D. (2009) Proceedings of the National Academy of the Sciences 106:298-303.
11. An empirical explanation of the speed-distance effect. Wojtach W.T., Sung K., Purves D. (2009) PLoS ONE 4(8): e6771.
12. The statistical structure of human speech sounds predicts musical universals. Schwartz DA, Howe CQ, Purves D, Journal of Neuroscience 23(18): 7160-7168. (2003).
13. Musical intervals in speech. Ross D, Choi J, Purves D, Proceedings of the National Academy of the Sciences 104(23): 9852-9857. (2007).
14. Major and minor music compared to excited and subdued speech. Bowling D.L., K.Gill. et al. Journal of the Acoustical Society of America 127(1): 491-503. (2010).
15. http://www.purveslab.net/research/
16. Natural scene statistics as the universal basis for color context effects. Long F, Purves D. Proceedings of the National Academy of the Sciences 100(25): 15190-15193. (2003).
17. Fechner G.T., Adler, H.E., Howes, D.H. & Boring, E.G., (1966) Elements of Psychophysics (Holt Rinehart and Winston, New York).
18. Shipley, W. C.; Nann, B. M.; Penfield, M. J. (1949). "The Apparent Length of Tilted Lines". Journal of Experimental Psychology. 39 (4): 548–551. doi:10.1037/h0060386. ISSN 0022-1015. PMID 18140138.
19. Howe and Purves (2002). PNAS 99(20): 13184-13188.
20. An empirical explanation of the flash-lag effect. Wojtach W.T., Sung K., Truong S., Purves D. (2008) Proceedings of the National Academy of the Sciences 105(2): 16338-16343.