Universal law of generalization

The universal law of generalization is a theory of cognition stating that the probability of a response to one stimulus being generalized to another is a function of the “distance” between the two stimuli in a psychological space. It was introduced in 1987 by Roger N. Shepard,[1][2] who began researching mechanisms of generalization while he was still a graduate student at Yale:

"I was now convinced that the problem of generalization was the most fundamental problem confronting learning theory. Because we never encounter exactly the same total situation twice, no theory of learning can be complete without a law governing how what is learned in one situation generalizes to another"[3]

Bird with earthworm: Shepard gives example of bird using "generalization," based on experience with one previous worm, to decide if another worm is edible.

Shepard’s 1987 paper gives a "generalization" example of a bird that has eaten one earthworm, and is presented with a slightly different-looking earthworm.[2] Explaining the concept of "psychological space" in the abstract of his 1987 paper, Shepard wrote:

"A psychological space is established for any set of stimuli by determining metric distances between the stimuli such that the probability that a response learned to any stimulus will generalize to any other is an invariant [monotonic function] of the distance between them"[2]

Using experimental evidence from both human and non-human subjects, Shepard hypothesized, more specifically, that the probability of generalization will fall off exponentially with the distance measured by one of two particular metrics. His analysis goes on to argue for the universality of this rule for all sentient organisms, due to evolutionary internalization.

Additional Research and Commentary

In 1988, Shepard received a response to his research from Daniel M. Ennis from the Philip Morris Research Center.[4] Ennis called into question whether Shepard’s theory was relevant as it echoed research already accomplished by Nosofsky, whose research presented  “several exceptions” [4] to Shepard’s theory. Shepard responded to him by asserting that Nosofsky’s experiments focused on “ the precise locations of individual stimuli in ‘psychological space’” [4] whereas his experiments focused on the “location, size, and shape of the region of psychological space corresponding to the set of stimuli having the same important consequence as a given training stimulus”.[4]  

Other researchers have followed up Shepard’s research with their own looks into the law of generalization. In 2000, Ken Cheng from Macquarie University [5] experimented on the special generalization of honeybees, comparing his findings with earlier research on humans and pigeons. Cheng explained his understanding of Shepard’s law of generalization in this study as such:

“Suppose that an animal finds food in a container at one location (S+). When the animal returns, the container is at a noticeably different location. Will the animal still ‘bet’ on finding food in the container? Underlying this question is the assumption that the animal can discriminate the two locations. Shepard’s law does not apply when the animal has trouble discriminating between stimuli. The question is whether the two locations have the same consequence of concern—in this case, whether the container contains food…”[5]

Cheng measured the bee’s response to copies of a container in different locations, the original staying in one area and holding sugar water. The copy containers would then either have plain tap water or sugar water. Measurements of generalization gradients were separated by both distance and direction according to Shepard’s law. Cheng found that “generalization gradients in both series were fitted by exponential functions, supporting Shepard’s law”.[5] Cheng’s research suggested that Shepard’s law had the potential to be generalized to more than only mammals and birds but to invertebrates as well. In 2001, Chater and Vitányi [6] attempted to provide a “mathematically more appealing form of the Universal Law”.[6]  They explain the necessity of a different mathematical outlook than the one presented in Shepard’s paper by illustrating the example of a photograph and its negative:

“Thus, while the positive and negative of the same picture are far away from each other in terms of Euclidean distance, they are at almost zero distance in terms of universal distance because interchanging the black and white pixels transforms one picture into the other” [6]

Chater and Vitányi admit that their way of approaching the law of generalization might be too abstract to fit a psychological equation, but argue that a simple abstract explanation would be as suitable as any other elementary explanation in a practical sense when faced with situations that are webbed with complex mathematical explanations.[6]

Steven A. Frank of the University of California [7] proposed another outlook on the approach of the law of generalization. He suggested that any established current mathematical scales used to measure generalization were unnecessarily detailed and that trying to keep generalization in a scale of constant values was not needed. Frank concluded that the exponential form of generalization could be accurately captured with shift invariance.

Chris R. Sims [8] has attempted to offer another look at the law of generalization through the scope of the principle of efficient coding. Sims makes the case for rate-distortion theory through perceptual identification experiments.[8]

References

1. "What your cell phone camera tells you about your brain". ScienceDaily.com. September 19, 2018. Retrieved February 5, 2019. A canonical law of cognitive science -- the Universal Law of Generalization, introduced in a 1987 article also published in Science -- tells us that your brain makes perceptual decisions based on how similar the new stimulus is to previous experience. Specifically, the law states that the probability you will extend a past experience to new stimulus depends on the similarity between the two experiences, with an exponential decay in probability as similarity decreases. This empirical pattern has proven correct in hundreds of experiments across species including humans, pigeons, and even honeybees.
2. Shepard, R. (1987-09-11). "Toward a universal law of generalization for psychological science". Science. 237 (4820): 1317–1323. doi:10.1126/science.3629243. ISSN 0036-8075.
3. Shepard, Roger N. (February 2004). "How a cognitive psychologist came to seek universal laws". Psychonomic Bulletin & Review. 11 (1): 1–23. doi:10.3758/bf03206455. ISSN 1069-9384.
4. Ennis, D. (1988-11-11). "Toward a universal law of generalization". Science. 242 (4880): 944–944. doi:10.1126/science.3187534. ISSN 0036-8075.
5. Cheng, Ken (September 2000). "Shepard's Universal Law Supported by Honeybees in Spatial Generalization". Psychological Science. 11 (5): 403–408. doi:10.1111/1467-9280.00278. ISSN 0956-7976.
6. Chater, Nick; Vitányi, Paul M.B. (June 2003). "The generalized universal law of generalization". Journal of Mathematical Psychology. 47 (3): 346–369. doi:10.1016/S0022-2496(03)00013-0.
7. Frank, Steven A. (2018-06-09). "Measurement invariance explains the universal law of generalization for psychological perception". dx.doi.org. Retrieved 2020-07-22.
8. Sims, Chris R. (2018-05-10). "Efficient coding explains the universal law of generalization in human perception". Science. 360 (6389): 652–656. doi:10.1126/science.aaq1118. ISSN 0036-8075.