Cooperative pulling paradigm

The cooperative pulling paradigm is an experimental design in which two or more animals pull rewards toward themselves via an apparatus that they cannot successfully operate alone. Researchers (ethologists, comparative psychologists, and evolutionary psychologists) use cooperative pulling experiments to try to understand how cooperation works and how and when it may have evolved.

A cooperative pulling experiment with dogs

The type of apparatus used in cooperative pulling experiments can vary. Researcher Meredith Crawford, who invented the experimental paradigm in 1937, used a mechanism consisting of two ropes attached to a rolling platform that was too heavy to be pulled by a single chimpanzee. The standard apparatus is one in which a single string or rope is threaded through loops on a movable platform. If only one participant pulls the string, it comes loose and the platform can no longer be retrieved. Only by pulling together in coordination can the participants be successful; success by chance is highly unlikely. Some researchers have designed apparatus that involve handles instead of ropes.

Although many animals retrieve rewards in their cooperative pulling tasks, the conclusions regarding cooperation are mixed and complex. Chimpanzees, bonobos, orangutans, capuchins, tamarins, wolves, elephants, ravens, and keas appear to understand the requirements of the task. For example, in a delay condition, the first animal has access to the apparatus before the other. If the animal waits for its partner before pulling, this suggests an understanding of cooperation. Chimpanzees, elephants, wolves, dogs, ravens, and keas wait; grey parrots, rooks, and otters fail to wait. Chimpanzees actively solicit help when needed. They appear to recall previous outcomes to recruit the most effective partner. In a group setting, chimpanzees punish initial competitive behavior (taking food without pulling, displacing animals) such that eventually successful cooperation becomes the norm.

As for the evolution of cooperation, evidence from cooperative pulling experiments provides support for the theory that cooperation evolved multiple times independently. The fact that basic characteristics of cooperation are present in some mammals and some birds points to a case of convergent evolution. Within social animals, cooperation is suspected to be a cognitive adaptation.

Background

Many species of animals cooperate in the wild.[1] Collaborative hunting has been observed in the air (e.g., among Aplomado falcons),[2] on land (e.g., among lions),[3] in the water (e.g., among killer whales),[4] and under the ground (e.g., among driver ants).[5] Further examples of cooperation include parents and others working together to raise young[1] (e.g., among African elephants),[6] and groups defending their territory, which has been studied in primates and other social species such as bottlenose dolphins, spotted hyenas, and common ravens.[7]

Researchers from various disciplines have been interested in cooperation in animals.[8] Ethologists study animal behavior in general.[9] Comparative psychologists are interested in the origins, differences, and commonalities in psychological capacities across animal species.[10] Evolutionary psychologists investigate the origin of human behavior and cognition, and cooperation is of great interest to them, as human societies are built on collaborative activities.[11][12]

For animals to be considered cooperating, partners must take account of each other's behavior to pursue their common goal. There are various levels of cooperation. These increase in temporal and spatial complexity from performing similar actions, to synchrony (similar actions performed in unison), then coordination (similar actions performed at the same time and place), and finally collaboration (complementary actions performed at the same time and place).[13][14] Researchers use controlled experiments to analyze the strategies applied by cooperating animals, and to investigate the underlying mechanisms that lead species to develop cooperative behavior.[1][8]

Method

The cooperative pulling paradigm is an experimental design in which two or more individuals, typically but not necessarily animals, can pull rewards towards themselves via an apparatus they can not successfully operate alone.[15] The cooperative pulling paradigm is the most popular paradigm for testing cooperation in animals.[16]

Apparatus
A loose-string apparatus: the platform will only slide into reach if both animals pull at the same time. If only one animal pulls the string it will come loose.[17]

The type of apparatus used in cooperative pulling experiments can vary. Researcher Meredith Crawford, who invented the experimental paradigm in 1937 while at the Yerkes National Primate Research Center, used an apparatus consisting of two ropes attached to a box that was too heavy to be pulled by a single chimpanzee.[18][19] The standard apparatus is used in the loose-string task, designed by Hirata in 2003, in which a single string or rope is threaded through loops on a movable platform. If only one participant pulls the string, it comes loose and the platform can no longer be retrieved.[20][21] Only by pulling together in coordination can the participants be successful; success by chance is highly unlikely.[22] Some researchers have designed apparatus that involve handles instead of ropes.[23][24] De Waal and Brosnan have argued that complex electronically-mediated devices are not conducive to arrive at findings regarding cooperation. This is in contrast to mechanical pulling devices, in which the animals can see and feel their pull having immediate effect.[25] String-pulling tasks have advantages in terms of ecological validity for animals that pull branches with food towards themselves.[26][upper-alpha 1] Tasks in which participants have different roles in collaboration, such as for example, one pulls a handle and the other one needs to insert a stick, are considered outside the cooperative pulling paradigm.[28]

Subjects

So far, fewer than twenty species have participated in cooperative pulling experiments: chimpanzees,[18] bonobos,[29] orangutans,[30] capuchin monkeys,[31] tamarins,[32] macaques,[33] humans,[12] hyenas,[34] wolves,[35] dogs,[35] elephants,[36] otters,[37] dolphins,[38] rooks,[39] ravens,[40] parrots,[41] and keas.[42][upper-alpha 2] Researchers have picked species that cooperate in the wild (e.g., capuchins),[45] live in social structures (e.g., wolves),[46] or have known cognitive abilities (e.g., orangutans).[30] Most of the participating animals have been in human care at an animal research center;[18][47] some lived semi-free at a sanctuary in their natural habitat.[48][49] One study involved free animals (Barbary macaques) in the wild.[50]

Conditions

To arrive at conclusions regarding cooperation, researchers have designed experiments with various conditions.[48][51][52]

Delay

The first animal has access to the apparatus before the other one. If the animal does not wait for its partner this suggests a lack of understanding of the requirements for successful cooperation.[48][53]

Recruitment

The subject recruits the partner (for example by opening a door) when the task requires cooperation[48].

Partner choice

The first animal gets to choose which animal from a pair it wants as a partner.[48] In some cases individual animals from within a group can decide to join an animal already at the apparatus.[7]

Apparatus choice

Instead of just one apparatus in the test area there are two identical ones. Animals can decide to work on the same one (which can lead to success) or on different ones (which will lead to failure).[54] A further design involves two different apparatus. The first animal can decide whether to use an apparatus that can be operated alone or one that requires and has a partner waiting.[55][56] A 'no rope' version involves an apparatus where everything is the same except for the rope on the partner's side being coiled up and not accessible to the partner.[57]

Reward

Rewards can be food split equally over two bowls in front of each animal, or in one bowl only. The type of food can vary from lots of small pieces to one big lump (e.g., slices of an apple vs. a whole apple).[58] In combination with the apparatus choice, the reward for the joint-task apparatus is often twice as big as the reward for the solo apparatus.[42] Another variation is a modified apparatus where one partner gets food before the other, requiring the first one to keep pulling despite already having received the reward.[59]

Visibility

Typically the animals can see each other, all rewards, and all parts of the apparatus. To assess the role of visual communication, sometimes an opaque divider is placed such that the animals can no longer see each other, but can still see both rewards.[60]

Training

Animals are often first trained with an apparatus that can be operated by one individual. For example, the two ends of a string are on top of each other and a single animal can pull both ends. A technique called shaping can be used by gradually extending the distance between the string ends,[21] or by gradually extending the length of delay between the arrival of the first and second animal at the apparatus.[61]

Findings

Overview

Although many animals retrieve rewards in their cooperative pulling tasks, the conclusions regarding cooperation are mixed and complex.[36][62] Some researchers have attributed successful cooperation to random simultaneous action,[52] or to the simple reactive behavior of pulling the rope when it moves.[63] Many trials with capuchins, hyenas, parrots and rooks led to failure because one partner pulled without the other present, suggesting a lack of understanding of cooperation.[64] A few researchers have offered the possible explanation that animals may understand cooperation to some extent but simply can not suppress the desire to have food they see.[65]

But there is evidence that some species do have an understanding of cooperation and perform intentional coordination to achieve a goal.[66][36] Specifically, chimpanzees,[48] bonobos,[67] orangutans,[23] tamarins,[68] capuchins,[69] elephants,[70] wolves,[46] ravens,[71] and keas[72] appear to understand how cooperation works.[73] Chimpanzees not only wait for a partner, but will actively solicit help when needed.[48] They appear to recall previous outcomes to recruit the most effective partner.[48] In a group setting, chimpanzees punish initial competitive behavior (taking food without pulling, displacing animals) such that eventually, after many trials, successful cooperation becomes the norm.[74] Bonobos, which are social animals with higher tolerance levels, can outperform chimpanzees on some cooperative tasks.[29] Elephants will wait for 45 seconds for a partner to arrive before they start a cooperative pulling task;[36] wolves do the same for 10 seconds.[46] Dogs raised as pets are also able to wait for a partner, albeit only for a few seconds;[75] pack dogs on the other hand rarely succeed in cooperative pulling in any condition.[76] Among birds, ravens are able to learn to wait after many trials,[71] while keas have set the record in waiting for a partner, 65 seconds.[72] Mere knowledge of the presence of a partner is not enough for success: when a barrier with a small hole was placed between two capuchins, obstructing the view of the partner's actions, the success rate dropped.[77] Of those species tested in the delay condition, parrots, rooks, and otters failed.[1][78]

In 2008, Seed, Clayton and Emery said the study of the proximate mechanisms underpinning cooperation in animals was in its infancy, due in part to the poor performances of animals such as chimpanzees in early tests that did not take factors such as inter-individual tolerance into account.[79] In 2006, Melis, Hare, and Tomasello had shown that the performance of chimpanzees in cooperative tasks was strongly influenced by levels of inter-individual tolerance [80].Several studies since have highlighted the fact that tolerance has a direct impact on cooperation success, as the more tolerant an animal is around food the better it performs.[81] Subordinate animals seem simply not willing to risk being attacked by intolerant dominant animals, even if it means they will not obtain food either. [80] In general, cooperation will not emerge if individuals can not share the spoils obtained through their joint effort.[82] Temperament, whether an animal is bold or shy, has also been found to predict success.[83]

As for the evolution of cooperation, evidence from cooperative pulling experiments appears to support the theory that cooperation evolved multiple times independently. The fact that basic characteristics of cooperation are present in some mammals and some birds points to a case of convergent evolution.[57] Within social animals, cooperation is suspected to be a cognitive adaptation.[84] The ability of humans to cooperate is likely to have been inherited from an ancestor shared with at least chimpanzees and bonobos.[85] The superior scale and range of human cooperation comes mainly from the ability to use language to exchange social information.[86]

Primates

Chimpanzees
Chimpanzees

Chimpanzees (Pan troglodytes) are smart, social animals.[12] In the wild they cooperate to hunt, dominate rival groups, and defend their territory.[87] They have participated in many cooperative pulling experiments.[88][89] The first ever cooperative pulling experiment involved captive chimpanzees. In the 1930s Crawford was a student and researcher at the Yerkes National Primate Research Center.[19] In 1937 he published a study of two young chimpanzees named Bula and Bimba pulling ropes attached to a box. The box was too heavy to be pulled in by just one ape. On top of the box was food. The two participants synchronized their pulling and were able to get the food reward in four to five short pulls. In a second part of the study, Crawford fed Bula so much prior to the test that she was no longer interested in the food reward. By poking her and pushing her hand towards the rope, Bimba tried to enlist her help in the task, with success.[19][18] In a follow-up experiment with seven pairs of chimpanzees Crawford found none of the apes spontaneously cooperated. Only after extensive training were they able to work together to obtain food. They also failed to transfer this new skill to a slightly different task, in which the ropes were hanging from the ceiling.[87][90]

Describing the 1930s black and white film Crawford and Nissen made of Crawford's cooperative pulling experiments with chimpanzees, professor Frans de Waal wrote: "I have shown a digitized version to many audiences, causing much laughter in recognition of the humanlike encouragements. People are quick to grasp the movie's essence: apes have a solid understanding of the advantages of cooperation."[19][upper-alpha 3]

Similar mixed results, not matching the cooperative abilities observed in chimpanzees in the wild, were obtained in later studies by other researchers using a variety of experimental set-ups, including the loose-string task pioneered by Hirata.[87][20] Povinelli and O’Neill, for example, found that trained chimpanzees were unable to teach naive chimpanzees to cooperate on a Crawford-like box-pulling task. The naive animals did not imitate the experts.[91][92] Chalmeau and Gallo found only two chimpanzees consistently cooperating in their handle-pulling task, and this involved one ape holding his own handle and waiting for the other to pull his. They concluded that social factors and not limited cognitive abilities were the reason for lack of widespread success, as they observed dominant chimpanzees controlling the apparatus and preventing others from interacting.[93]

Melis, Hare, and Tomasello set up an experiment to control for such social factors. In a loose-string cooperative task without training they compared the ability of pairs of captive chimpanzees who in a non-cooperative setting were willing to share food with each other to pairs who were less inclined to do so. The results showed that food sharing was a good predictor for success in the cooperative pulling task. Melis, Hare, and Tomasello concluded that mixed results in the past could at least partially be explained by a failure to control for such social constraints.[87] In a follow-up study with semi–free-ranging chimpanzees, again using the loose-string task, the researchers introduced the delay task, in which subjects were tested in their ability to wait for the partner. After mastering this task, they participated in a new task designed to measure their ability to recruit the partner. They found that the apes only recruited a partner (by unlocking a door) if the task required cooperation. When given the choice between partners, the apes chose the more effective one, based on their experience with each of them previously.[48][upper-alpha 4]

Suchak, Eppley, Campbell, Feldman, Quarles, and de Waal argued that even when experiments take social relationships into account, the results still do not match the cooperation capabilities observed in the wild.[74] They set out to increase the ecological validity of their experiments by placing a handle-pulling apparatus in an open-group setting, allowing the captive chimpanzees themselves to choose to interact with it or not, and with whom. They also refrained from any training, offered as little human intervention as possible, and extended the duration to much longer than any test had ever done, to 47 days of 1 hour tests.[88][24] The chimpanzees first discovered that cooperation could lead to success, but as more individuals became aware of this new way to obtain food, competition increased, taking the form of dominant apes displacing others, monopolizing the apparatus, and freeloading: taking the food others worked for. This competition led to fewer successful cooperative acts.[95] The group did manage to restore and increase levels of cooperative behavior by various enforcement techniques: dominant individuals were unable to recruit partners and abandoned the apparatus, displacement was met with aggressive protest, and freeloaders were punished by third-party arbiters.[96] When the researchers repeated this experiment with a brand new group of chimpanzees who not yet had established a social hierarchy, they again found that cooperation overcame competition in the long run.[97] In a later study with a mix of novices and experts, Suchak, Watzek, Quarles, and de Waal found that novices learned rapidly in the presence of experts, although likely with limited understanding of the task.[98]

Greenberg, Hamann, Warneken, and Tomasello used a modified apparatus that required two captive chimpanzees to pull, but delivered food to one ape first. They found that in many trials the apes who already had received a reward from joint effort kept pulling to help their partner obtain their food.[99] These partners did not need to gesture to solicit help, suggesting there was an understanding of what was wanted and needed.[100]

Bonobos
Bonobos

Bonobos (Pan paniscus) are social animals that live in less hierarchical structures than chimpanzees. Hare, Melis, Woods, Hastings, and Wrangham set out to compare cooperation in chimpanzees and bonobos. They first ran a cofeeding experiment for each species. Pairs of bonobos were given two food dishes. In some trials both dishes had sliced fruit; in some one dish was empty and the other had sliced fruit; and in some one dish was empty and the other contained just two slices of fruit. The same set-up was then used for pairs of chimpanzees. When both dishes had food, there was no difference in behavior between bonobos and chimpanzees. But when only one dish contained food, bonobos were more than twice as likely to share food than chimpanzees. Bonobos were more tolerant of each other than chimpanzees.[67] The researchers then ran a loose-string cooperation task with both dishes filled with sharable food. The results showed similar success rates for bonobos and chimpanzees, 69% of chimpanzee pairs and 50% of bonobo pairs spontaneously solving the task at least once within the six-trial test session.[101]

In a third experiment, a year later, the same cooperation task was administered but now with different food distributions. The bonobos outperformed the chimpanzees in the condition where one dish only had food and the food was clumped making it easier to monopolize the food reward. Bonobos cooperated more often in this condition. On average a single chimpanzee partner monopolized food rewards more often than a single bonobo did. In the condition where both dishes were filled with food, chimpanzees and bonobos performed similarly, as they had done the year before. The researchers concluded that the differences in performance between species were not due to differences in age, relationships, or experience.[102] It was the bonobos' higher social tolerance level that enabled them to outperform their relatives.[67]

Orangutans
Orangutan

Orangutans (Pongo pygmaeus) are tool-using apes that are mostly solitary.[103][104] Chalmeau, Lardeux, Brandibas, and Gallo tested the cooperative capabilities of a pair of orangutans, using a device with handles. Only through simultaneous pulling could the pair retrieve a food reward. Without any training the orangutans succeeded in the first session. Over the course of 30 sessions, the apes succeeded more quickly, having learned to coordinate. Across trials the researchers found an increase in a sequence of actions that suggested understanding of cooperation: first looking at the partner; then if the partner holds or pulls the handle, starting to pull.[105]

The researchers also concluded that the orangutans learned a partner had to be present for success.[23] For example, they observed that time spent alone at the apparatus decreased as the trials progressed.[106] In some instances one orangutan pushed the other towards the free handle, soliciting cooperation.[107] The researchers observed an asymmetry: one ape did all the monitoring and coordinating, the other one seemed to simply pull if the first one was present.[108] Rewards did not have to be shared equally for success to appear, as one orangutan took 92% of all food. This ape anticipated the falling of food and stuck his hand out first, before recruiting help from his partner.[109] Chalmeau, Lardeux, Brandibas, and Gallo concluded the apes appeared to understand the requirements of the cooperative task.[23]

Capuchins
Capuchin monkeys

Capuchins (Sapajus apella) are large-brained monkeys that sometimes hunt cooperatively in the wild and show, for nonhuman primates, unusually high levels of social tolerance around food.[45][110] Early experiments to prove their ability to cooperate were unsuccessful. These tests involved capuchins having to pull handles or press levers in complex devices that the animals did not understand.[25][111] They did not pull the handle more often when a partner was pulling; both novices and experienced participants kept pulling even in situations where success was impossible. Visalberghi, Quarantotti, and Tranchida concluded that there was no evidence of an appreciation of the role played by the partner.[112]

The first test with evidence of cooperation in capuchins happened when de Waal and Brosnan adopted Crawford's pulling paradigm. Two captive monkeys were situated in adjacent sections of a test chamber, with a mesh partition between them. In front of them was an apparatus consisting of a counter-weighted tray with two pull bars and two food cups. Each monkey had access to only one bar and one food cup, but could see both, and only one cup was filled with food. The tray was too heavy for one monkey to pull it in, with weights established over trials lasting three years. Only when they worked together and both pulled could they move the tray, enabling one of them to grab the food. Trained monkeys were much more successful if they both obtained rewards after pulling than if only one of them received rewards. The pull rate dropped significantly when monkeys were alone at the apparatus, suggesting an understanding of the need for a partner.[69] In later tests, researchers replaced the mesh partition with an opaque barrier with a small hole, so that the monkeys could see the other one was there but not their actions. This dramatically reduced success in cooperation.[77][113]

Sammy was in such a hurry to collect her rewards that she released the tray before Bias had a chance to get hers. The tray bounced back, out of reach of Bias. While Sammy munched on her food, Bias threw a tantrum. She screamed her lungs out for half a minute until Sammy approached her pull bar again. She then helped Bias bring in the tray a second time. Sammy did not do so for her own benefit, since by now her own cup was empty. Sammy's corrective response seemed the result of Bias’s protest against the loss of an anticipated reward. This example shows cooperation, communication, and the fulfillment of an expectation, perhaps even an obligation.

 – Frans de Waal, 2006[114]

De Waal and Berger used the cooperative pulling paradigm to investigate animal economics. They compared the behavior when both transparent bowls were loaded with food to when just one was loaded, and with a solo task where the partner was only an observer and unable to help. They found that captive capuchin monkeys were willing to pull even if their bowl was empty and it was uncertain if their partner would share food. In 90% of cases the owner of the food did indeed share the food. Food was shared more often if the partner actually worked for it than just being an observer.[115]

Brosnan, Freeman, and de Waal tested captive capuchin monkeys on a bar-pulling apparatus with unequal rewards. Contrary to their expectations, rewards did not have to be distributed equally to achieve success. What mattered was the behavior in an unequal situation: pairs that tended to alternate which monkey received the higher-value food were more than twice as successful in obtaining rewards than pairs in which one monkey dominated the higher-value food.[116]

Tamarins
Cotton-top tamarins

Cottontop tamarins (Saguinus oedipus) are small monkeys who take care of their young cooperatively in the wild.[117] Cronin, Kurian, and Snowdon tested eight captive cottontop tamarins in a series of cooperative pulling experiments.[68] Two monkeys were put on opposite sides of a transparent apparatus containing food. Only if both monkeys pulled a handle on their side of the apparatus towards themselves at the same time would food drop down for them to obtain.[117] The tamarins were first trained, through shaping techniques, to use the handles successfully by themselves.[118] In the joint pulling test pairs were successful in 96% of trials.[119]

The researchers then ran a second study in which a tamarin was tested alone.[120] The results showed that tamarins pulled the handles at a lower rate when alone with the apparatus than when in the presence of a partner.[121] Cronin, Kurian, and Snowdon concluded from this that cottontop tamarins have a good understanding of cooperation.[68] They suggest that cottontop tamarins have developed cooperative behavior as a cognitive adaptation.[84]

Macaques
Barbary macaques

Molesti and Majolo tested a group of wild Barbary macaques (Macaca sylvanus) in Morocco to see if they would cooperate, and if so, what determined their partner choice. Macaques live in complex social environments and are relatively tolerant socially. After solo training, the researchers presented a loose-string apparatus for the cooperative task, which the animals were free to use.[122] Most animals that passed solo training were successful in spontaneously cooperating to obtain food (22 out of 26). More than half the pairs that chose to cooperate were juvenile-adult pairs. More than two monkeys pulling was never observed; stealing food from a partner was rare.[123] After a first successful cooperation, they were more likely to pull when a partner was directly available, but this was not always the case.[22] Molesti and Majolo did not rule out that pulling while no one held or pulled the other end of the rope was simply a signal to actively recruit a potential partner.[65] The researchers randomly introduced control trials in which the solo apparatus was set up as well. The macaques preferred to get the food alone when a partner was not needed during the control.[65]

The extent to which a monkey tolerated another was a good predictor for initiating cooperation. An individual was also found to be more successful with partners with whom they had a strong social bond. Pairs sharing a similar temperament were more likely to initiate cooperation. The quality of the relationship seemed to play an important role in the maintenance of cooperation over time.[65]

Humans

Rekers, Haun, and Tomasello tested the cooperation abilities and preferences of humans (Homo sapiens) and compared them to chimpanzees.[12] The researchers provided 24 three-year-old children with some basic training in pulling food rewards towards themselves; in pairs using a loose-string setup, and solo training in which the two ends of a rope were tied together. They then tested the children in an apparatus choice set-up. On one side was the loose end of a rope that threaded through the apparatus to the other child. On the other side were two ends of a rope that when pulled would pull a platform towards both the child and their partner. Both the joint-operator platform and the solo-operated platform were holding two food dishes, all containing the same amounts of food. That is, from a partner's perspective, on one side the child had to pull to get food; on the other the partner could get food without any effort. The children chose the joint-operated board in 78% of trials.[124]

The researchers then changed the design to ascertain if this choice preference was due to wishing to avoid freeloading and it may be that the children did not like their partner getting food without making any effort. In the modified set-up the partners never received any reward, not from the joint-operated apparatus and not from the solo-operated apparatus. Children again chose the joint-operated platform significantly more often, in 81% of trials. As in the first study, there was no significant difference in the time taken to obtain the food reward between using one side or the other. These results suggest that to obtain food, children prefer to work together with a partner as opposed to working alone.[124] The chimpanzees in their study appeared to choose between the two platforms randomly, indicating no preference to work collaboratively.[124] However, Bullinger, Melis, and Tomasello showed that chimpanzees actually exhibit a preference for working alone, unless cooperation is associated with higher pay-offs [56].

Other mammals

Hyenas
Spotted hyena

Captive spotted hyenas (Crocuta crocuta), social carnivores that hunt in groups, have cooperated to obtain food rewards by pulling ropes in an experimental setting.[13] Mimicking the natural choice hunting hyenas face when deciding which of many prey to jointly attack, researchers Drea and Carter set up two devices instead of one, as previously used in all cooperative pulling tasks with other species. With four ropes to pull from, the animals had to pick the two belonging to the same device to be successful.[51] If two vertically suspended ropes were simultaneously tugged, a spring-controlled trap door of an elevated platform was opened and previously hidden food dropped to the floor.[125] Another innovation was the introduction of more than two animals. One of the many factors the researchers controlled for was the Clever Hans effect (an effect in which humans unwittingly provide cues to animals), which they did by removing all humans from the test and by recording experiments on video.[126]

After extensive solo trials, all hyenas were successful in cooperating, displaying remarkable efficiency even on their first try.[127] On average, hyenas pulled on ropes more often when their companion was nearby and available to fulfil its partnership role.[128] With only a few solo trials, the success rate of the cooperation task was very low for pairs. In groups of four hyenas, all trials were successful, regardless of the number of reward platforms. Thereafter, group exposure to a cooperation task had enhancing effects on pairwise performance.[126] Social factors such as group size and hierarchy played a role. For instance, groups with a dominant member were far less successful than groups without, and lower-ranking animals were faster and consistently successful.[129] When pairing experienced cooperators with animals new to the cooperation task, the researchers found that experienced animals monitored the novices and modified their behavior to achieve success.[128] Despite initial accommodation, the pattern of rank-related social influences on partner performance also appeared in these tests with novices.[130]

Dogs
Labrador Retrievers (one of many breeds used in experiments)

Ostojić and Clayton administered the loose-string cooperation task to domestic dogs (Canis familiaris). Pet dogs first were given a solo task in which the string ends were close enough for one dog to pull at both. Then they were given a transfer test to assess if they could generalize their newly learned rule to novel situations. Finally, the joint task was administered. Dog pairs always came from the same household. In half of the joint tasks one of the pair of dogs was shortly delayed by an obstacle course.[53] All dogs that learned to master the solo task solved the joint task within 60 trials.[131] In the delayed condition, the not-delayed dog waited before pulling most of the time, but only for a few seconds. The researchers also tested dog–human pairs, again in delayed and not-delayed conditions. Dogs were equally successful when working with humans in the non-delayed condition, but far less successful when they had to wait for the human, who on average arrived with a 13-seconds longer delay than the delayed dog in the dog–dog trials. Ostojić and Clayton concluded that inhibiting the necessary action was not easy for dogs. They ruled out that dogs simply went for any moving string, as in the dog–human trials the humans did not pull hard enough to make the other end move.[132] They attributed success to the dogs' ability to read the social cue of their partner's behavior, but could not rule out that visual feedback of seeing rewards incrementally move closer also played a role.[133]

These results with pet dogs stand in stark contrast to the results with pack dogs, which in a study by Marshall-Pescini, Schwarz, Kostelnik, Virányi, and Range rarely succeeded in obtaining food. The researchers theorized that pet dogs are trained not to engage in conflicts over resources, promoting a level of tolerance, which may facilitate cooperation. The pack dogs were used to competition over resources and thus were likely to have conflict avoidance strategies, which constrain cooperation.[134]

Wolves
Gray wolf

Marshall-Pescini, Schwarz, Kostelnik, Virányi, and Range set out to test two competing hypotheses regarding cooperation in wolves (Canis lupus) and dogs. On the one hand, it could be theorized that dogs have been selected, during domestication, for tame temperaments and an inclination to cooperate and therefore should outperform wolves on a cooperative pulling task. On the other hand, it could be argued that dogs have evolved to become less able to work jointly with other dogs because of their reliance on humans. Wolves rely on each other for hunting, raising young and defending their territory; dogs rarely rely on other dogs.[76] The researchers set up a cooperative pulling task for captive wolves and pack dogs. Without any training on this task, five of the seven wolf pairs were successful at least once, but only one dog pair out of eight managed to obtain food, and only once.[54] After solo training, again the wolves far outperformed the dogs on the joint task. The researchers concluded that the difference does not stem from a difference in understanding of the task (their cognitive capabilities are largely the same), nor from a difference in social aspects (for both species, aggressive behavior by dominant animals was rare, as was submissive behavior by lower ranked ones). More likely is that dogs avoid potential conflict over a resource more than wolves do, something which has been observed in other studies as well.[134]

The wolves, but not the dogs, were then tested in pairs in a set-up with two identical apparatus 10 meters (39 ft) apart, requiring them to coordinate in time and space. In 74% of the trials they succeeded. The stronger the bond between the partners and the smaller the distance in rank, the better they performed.[46] In a subsequent delay condition, with the second wolf released 10 seconds after the first, most wolves did well, one being successful in 94% of trials.[46]

Elephants
Asian elephants

Elephants have a complex social structure and large brains that enable them to solve many problems.[52] Their size and strength do not make them easy candidates for experiments. Researchers Plotnik, Lair, Suphachoksahakun, and de Waal adapted the apparatus and task to elephant requirements. They trained captive Asian elephants (Elephas maximus) to use a rope to pull a sliding platform with food on it towards themselves. Once the elephants managed this solo task, the researchers introduced a loose-string apparatus by threading the rope around the platform. At first, two elephants were released simultaneously to walk side by side in two lanes to the two loose ends of the rope. Using their trunks the animals coordinated their actions and retrieved the food.[61]

At this stage they could simply be applying a 'see the rope, pull the rope' strategy. To see whether they understood the requirements of the task the researchers introduced a delay for one elephant, initially of 5 seconds and ultimately of 45 seconds. At first the lead elephant failed to retrieve the food but was soon seen to wait for a partner. Across 60 trials the first elephant waited for the second one before pulling in most cases.[61] In a further control the researchers prevented the second elephant from being able to access its end of the rope. In almost all of these cases the first elephant did not pull the rope, and four of the six returned when they saw the other rope end was not going to be accessible to their partner. The researchers concluded that this suggested the elephants understood they needed their partner to be present and to have access to the rope to succeed.[135] One elephant never pulled the rope but simply put her foot on the rope and let the partner do all the pulling. Another one waited for his partner's release at the starting line rather than waiting at the rope.[136] Plotnik, Lair, Suphachoksahakun, and de Waal conceded that it is difficult to distinguish learning from understanding. They did prove that elephants show a propensity towards deliberate cooperation. The speed with which they learned the critical ingredients of successful cooperation puts them on par with chimpanzees and bonobos.[70]

Otters
Set-up in the delay conditions of Schmelz et al.'s experiments with otters

Schmelz, Duguid, Bohn, and Völter presented two species of captive otters, giant otters (Pteronura brasiliensis) and Asian small-clawed otters (Aonyx cinerea), with the loose-string task.[137] Both species raise young cooperatively and live in small groups. Because giant otters forage together but small-clawed otters do not, the researchers expected the giant otters to do better in the cooperative pulling experiment.[138] After solo training, they tested both species in a group setting, to maintain ecological validity.[139] The results showed that most pairs of otters were successful in pulling food rewards to themselves. Contrary to expectation, there was no difference between the species in success rate.[140] In a subsequent experiment the researchers first lured the group away from the apparatus into the opposite corner of the enclosure. Then they put food on the apparatus and observed what happened when the first otter arrived at the nearest end of the rope, as there was no partner yet at the other end. Very few trials led to success in this condition as otters pulled the rope as soon as they could. The researchers concluded from this that the otters did not understand the necessary elements of successful cooperation, or, alternatively, they understood but were unable to inhibit the desire to reach for the food. When the same task was repeated with a longer rope, success rate did go up, but the otters appeared unable to learn from this and be successful in the next task with the rope length restored to the original length.[78] Schmelz, Duguid, Bohn, and Völter suggested that an understanding of cooperation may not be required for successful cooperation in the wild. Cooperative hunting may be possible through situational coordination and mutualism, without any complex social cognitive abilities.[141]

Dolphins
Common bottlenose dolphin

Two groups of researchers (first Kuczaj, Winship, and Eskelinen, and then Eskelinen, Winship, and Jones) adapted the cooperative pulling paradigm for captive bottlenose dolphins (Tursiops truncatus).[38][142] As apparatus they used a container which could only be opened at one end if two dolphins each pulled a rope on either end. That is, the dolphins would have to face each other and pull in opposite directions.[143] They first attached the container to a stationary dock so a single dolphin could learn to open it and get the food reward. Then they ran trials in which the container was free floating in a large test area with six dolphins. In Kuczaj, Winship, and Eskelinen's study, only two dolphins interacted with the container. In eight of the twelve trials they pulled simultaneously and obtained food. Once, they also managed to open the container through asynchronous pulling, and once a single male dolphin managed to open it by himself.[144] Kuczaj, Winship, and Eskelinen admitted that this behavior may appear to be cooperation but could possibly be competition. They conceded it is possible that the dolphins did not understand the role of the other dolphin, but instead simply tolerated it pulling on the other side.[145] King, Allen, Connor, and Jaakkola later argued that this design makes for a competitive ‘tug-of-war’, not cooperation, and any conclusions regarding cooperation should therefore be invalid.[146]

Birds

Rooks
Rook

Rooks (Corvus frugilegus) are large-brained members of the bird family Corvidae. They live in big groups and have a high level of social tolerance.[21] Researchers Seed, Clayton, and Emery set up a loose-string experiment with eight captive rooks. They were first trained in a solo task, with the string ends placed at 1 cm, 3 cm and ultimately 6 cm apart (0.4, 1.2, and 2.4 inch respectively).[21] A pair's willingness to share food was then tested, and was found to differ somewhat between pairs, although food was rarely monopolized by a dominant bird. In the cooperative task, all pairs were able to solve the cooperation problem and retrieve food; two pairs managed this in their first session.[147] Food sharing was a good predictor for successful cooperation.[148]

In a subsequent delay test, where one partner had access to the apparatus first, all rooks pulled the string without waiting for their partner to enter the test area in the majority of trials.[55] In a second variant, birds were given a choice between a platform they could operate successfully alone and one that required a pulling partner. When tested alone, four of the six rooks showed no significant preference for either platform.[55] Seed, Clayton, and Emery concluded that although successful at the cooperation task, it seemed unlikely that the rooks had an understanding of when cooperation was necessary.[149]

Researchers Scheid and Noë subsequently found that successful cooperation in rooks depended to a large extent on their temperament.[83] In their loose-string experiment with 13 captive rooks they distinguished between bold and shy animals.[150] The results were mixed, ranging from some pairs cooperating successfully every time to some pairs never cooperating.[151] In 81% of cases a rook should have waited for a partner, but it did not and started pulling.[152] Scheid and Noë concluded their experiment provided no evidence for or against rooks having an understanding of the task.[153] They attributed any cooperation success to common external cues and not coordination of actions. But all subjects did better when they were paired with a bolder partner.[152] The researchers suggested that in evolution, cooperation can emerge because bolder individuals encourage a risk-averse one to engage.[154]

Ravens

Massen, Ritter, and Bugnyar investigated the cooperative capabilities of captive common ravens (Corvus corax),[7] a species that frequently cooperates in the wild.[155] They found that without training ravens cooperated in the loose-string task.[7] The animals did not seem to pay attention to the behavior of their partners while cooperating, and, like rooks, did not seem to understand the need for a partner to be successful.[156] Tolerance of their partner was a critical factor for success. In one condition the researchers let ravens choose a partner from a group to cooperate with. Overall success was higher in this condition, and again, individuals that tolerated each other more had more success. The ravens also paid attention to reward distribution: they stopped cooperating when being cheated upon.[7]

Asakawa-Haas, Schiestl, Bugnyar, and Massen subsequently ran an open-choice experiment with eleven captive ravens in a group setting, using nine ravens from one group and two newcomers.[157] They found that the ravens' decision which partner to cooperate with was based on tolerance of proximity and not on whether they were part of the group or not.[155] The ravens in this experiment learned to wait for their partner and inhibit pulling the string too soon.[71]

Grey parrots
Grey parrot

Researchers Péron, Rat-Fischer, Lalot, Nagle and Bovet had captive grey parrots (Psittacus erithacus) try to cooperate in a loose-string experimental set-up. The grey parrots were able to act simultaneously but, like the rooks, largely failed to wait for a partner in the delay task. They did not make any attempts to recruit a helping partner.[1] The parrots did take the presence of a partner into account, since they all pulled more when a partner was present, but this could be explained by instrumental learning rather than a real understanding of the task.[158] The researchers also gave the parrots a choice between two apparatus, one from the solo task and one from the loose-string task, now stacked with double the food per bird. Two of the three parrots chose the solo apparatus when alone, and two of the three parrots preferred the joint-task apparatus when tested with a partner.[159] When paired up, social preferences and tolerance affected the likelihood a pair cooperated.[159]

Keas
The delayed partner arrival experiment with two keas

Keas (Nestor notabilis), parrots native to New Zealand, are a distant relative of the grey parrot.[160] They live in complex social groups and do well on cognitive tests.[161] Heaney, Gray, and Taylor gave four captive keas a series of cooperative loose-string tasks. After solo training and shaping with string ends increasingly further apart, two birds were released simultaneously in a joint loose-string task. Both pairs did very well, one pair failing only 5 in 60 trials.[162] Shaping was then used in a delay task, with the partner released after one second, then two, and gradually up to 25 seconds later than the first bird.[161] The birds managed to wait for a partner between 74% to 91% of test trials, including success at 65 seconds delay, longer than any other animal of any species had been tested for.[161] To assess if this success could be explained by the learning of a combination of cues, such as seeing a partner while feeling tension on the string, or by a proper understanding of cooperation, the researchers randomly gave the keas a set-up they could solve alone or one in which they needed to cooperate with a delayed partner. Three of the four keas were successful at a significant rate: they chose to wait when they had to and immediately pulled when the task could be done alone.[163] However, when the researchers modified the set-up and coiled up the string end of the delayed partner, no bird was successful at discriminating between a duo platform with both ends of string available to both keas and a duo platform with the partner's string coiled out of reach. The researchers were not able to determine the reason for this result. They speculated it could be that keas do have an understanding of when they need a partner but do not have a clear idea of the role their partner plays in relation to the string, or they may lack of a full causal understanding of how the string works.[57] Finally, the researchers attempted to ascertain if keas have a preference for working alone or together. No preference was found in three of the four keas, but one kea preferred the duo platform significantly more.[164] Heaney, Gray, and Taylor concluded that these results put keas on a par with elephants and chimpanzees in terms of cooperative pulling.[72]

These conclusions are in sharp contrast to those of Schwing, Jocteur, Wein, Noë, and Massen, who tested ten captive keas in a loose-string task on an apparatus that provided limited visibility to follow the trajectory of the string.[165] After training with a human partner (no solo training was done), only 19% of trials led to the birds obtaining food in the joint task. The researchers found that the closer the birds were affiliated, the more successful they were in the cooperation task. The keas did not seem to understand either the mechanics of the loose-string apparatus or the need of a partner, as in training with humans they still pulled the string even when the human was too far away or facing the wrong way. The way rewards were distributed had a small effect on the likelihood of cooperation attempts. The difference in social rank or dominance did not seem to matter.[166]

Footnotes
  1. Pulling a string does require some level of cognition. It is unlikely to be governed fully by innate processes, as examples exist of animals that feed with their feet but use various techniques to pull a string.[27]
  2. Werdenich and Huber investigated cooperation in marmosets (Callithrix jacchus) using an experimental set-up where only one monkey pulled instead of both.[43] They first trained eight marmosets in a solo task to pull a handle to bring a food reward within reach. Next, 16 pairs were given the cooperation test with a modified apparatus such that it required one monkey, the producer, to pull a handle so the other one, the scrounger, could grab a bowl with food, a single reward. All marmosets were willing and able to cooperate at least once. But only half of all pairings solved the task; primarily pairs in which the dominant monkey was the scrounger were successful. The researchers determined that tolerance of higher-ranking marmosets was an important factor in cooperation success.[43] Marmosets are cooperative breeders with cognitive abilities far lower than apes.[44]
  3. For fifty years the film recordings of Crawford's experiments were archived in a box in the old library at Yerkes, thus remaining unseen, until de Waal and colleagues stumbled upon it in a clean-up operation. They struggled to find a projector that could play the film.[19]
  4. Vail, Manica, and Bshary showed that, like chimpanzees, coral trout (Plectropomus leopardus) too can choose appropriately when and with whom to collaborate. They found this in the context of a set-up of a collaborative hunting relationship with moray eels, using experiments analogous to cooperative pulling tasks with chimpanzees but modified to be ecologically relevant to trout.[94]
gollark: There are several thousand.
gollark: In 24 minutes? Impressive.
gollark: "Vinyl' is down. Perhaps another music bot is needed.
gollark: I will join by 2030.
gollark: Implausible. This would be hard to do.

References

Notes
  1. Péron et al. 2011, p. 545.
  2. Hector 1986, p. 247.
  3. Stander 1992, p. 445.
  4. Pitman & Durban 2012, p. 16.
  5. Brian 2012, p. 18.
  6. Lee 1987, p. 278.
  7. Massen, Ritter & Bugnyar 2015, p. 1.
  8. Noë 2006, p. 1.
  9. de Waal 2016, p. 9.
  10. Papini 2003, p. 211.
  11. Cosmides, Tooby & Barkow 1992, pp. 9,163.
  12. Rekers, Haun & Tomasello 2011, p. 1756.
  13. Drea & Carter 2009, p. 967.
  14. Boesch & Boesch 1989.
  15. de Waal 2016, p. 276.
  16. Ostojić & Clayton 2014, p. 445.
  17. Asakawa-Haas et al. 2016, p. 4.
  18. Crawford 1937.
  19. de Waal 2016, p. 185.
  20. Hirata 2003.
  21. Seed, Clayton & Emery 2008, p. 1422.
  22. Molesti & Majolo 2016, p. 142.
  23. Chalmeau et al. 1997, p. 23.
  24. Suchak et al. 2014.
  25. de Waal & Brosnan 2005, p. 95.
  26. Molesti & Majolo 2016, p. 134.
  27. Jacobs & Osvath 2015, p. 111.
  28. Melis & Tomasello 2013.
  29. Hare et al. 2007.
  30. Chalmeau et al. 1997.
  31. de Waal & Brosnan 2005.
  32. Cronin, Kurian & Snowdon 2005.
  33. Molesti & Majolo 2016.
  34. Drea & Carter 2009.
  35. Marshall-Pescini et al. 2017.
  36. Plotnik et al. 2011.
  37. Schmelz et al. 2017.
  38. Kuczaj, Winship & Eskelinen 2015.
  39. Scheid & Noë 2010.
  40. Asakawa-Haas et al. 2016.
  41. Péron et al. 2011.
  42. Heaney, Gray & Taylor 2017.
  43. Werdenich & Huber 2002, p. 771.
  44. Burkart et al. 2007, p. 19765.
  45. de Waal & Berger 2000, p. 563.
  46. Marshall-Pescini et al. 2017, p. 11795.
  47. Noë 2006, p. 5.
  48. Melis, Hare & Tomasello 2006b, p. 1297.
  49. Hare 2007, p. 64.
  50. Molesti & Majolo 2016, p. 133.
  51. Drea & Carter 2009, p. 969.
  52. Plotnik et al. 2011, p. 5116.
  53. Ostojić & Clayton 2014, pp. 446–447.
  54. Marshall-Pescini et al. 2017, p. 11794.
  55. Seed, Clayton & Emery 2008, p. 1426.
  56. Bullinger, Melis & Tomasello 2011, p. 1135.
  57. Heaney, Gray & Taylor 2017, p. 11.
  58. Seed, Clayton & Emery 2008, p. 1423.
  59. Greenberg et al. 2010, p. 876.
  60. Brosnan & de Waal 2002, p. 145.
  61. Plotnik et al. 2011, pp. 5116–5117.
  62. Albiach-Serrano 2015, p. 361.
  63. Heaney, Gray & Taylor 2017, p. 5.
  64. Jelbert et al. 2015, p. 2.
  65. Molesti & Majolo 2016, p. 143.
  66. Jelbert et al. 2015, p. 1.
  67. Hare et al. 2007, p. 619.
  68. Cronin, Kurian & Snowdon 2005, p. 1.
  69. de Waal & Brosnan 2005, pp. 95–96.
  70. Plotnik et al. 2011, p. 5120.
  71. Asakawa-Haas et al. 2016, p. 9.
  72. Heaney, Gray & Taylor 2017, p. 1.
  73. Jelbert et al. 2015, pp. 1–2.
  74. Suchak et al. 2016, p. 10218.
  75. Ostojić & Clayton 2014.
  76. Marshall-Pescini et al. 2017, p. 11793.
  77. de Waal & Suchak 2010, p. 2713.
  78. Schmelz et al. 2017, p. 1111.
  79. Seed, Clayton & Emery 2008.
  80. Melis, Hare & Tomasello 2006a.
  81. Péron et al. 2011, p. 546.
  82. Melis & Semmann 2010, p. 2669.
  83. Scheid & Noë 2010, p. 545.
  84. Cronin, Kurian & Snowdon 2005, p. 11.
  85. Suchak et al. 2016, p. 10215.
  86. Melis & Semmann 2010, p. 2671.
  87. Melis, Hare & Tomasello 2006, p. 275.
  88. Suchak et al. 2016, p. 10216.
  89. Hirata & Fuwa 2007, p. 13.
  90. Crawford 1941.
  91. Povinelli & O’Neill 2000.
  92. Melis, Hare & Tomasello 2006, pp. 275–276.
  93. Melis, Hare & Tomasello 2006, p. 276.
  94. Vail, Manica & Bshary 2014.
  95. Suchak et al. 2016, p. 10217.
  96. Suchak et al. 2016, pp. 10217–10218.
  97. Suchak et al. 2016, p. 10219.
  98. Suchak et al. 2018, p. 87.
  99. Greenberg et al. 2010, p. 873.
  100. Greenberg et al. 2010, p. 879.
  101. Hare et al. 2007, p. 620.
  102. Hare et al. 2007, p. 621.
  103. de Waal 2016, p. 81.
  104. Te Boekhorst, Schürmann & Sugardjito 1990.
  105. Chalmeau et al. 1997, p. 26.
  106. Chalmeau et al. 1997, p. 27.
  107. Chalmeau et al. 1997, pp. 27–28.
  108. Chalmeau et al. 1997, p. 29.
  109. Chalmeau et al. 1997, p. 30.
  110. de Waal 2006, p. 355.
  111. Chalmeau, Visalberghi & Gallo 1997.
  112. Visalberghi, Quarantotti & Tranchida 2000, p. 300.
  113. Mendres & de Waal 2000.
  114. de Waal 2006, p. 358.
  115. de Waal 2006, pp. 355–356.
  116. Brosnan, Freeman & de Waal 2006, p. 713.
  117. Cronin, Kurian & Snowdon 2005, p. 3.
  118. Cronin, Kurian & Snowdon 2005, p. 4.
  119. Cronin, Kurian & Snowdon 2005, p. 6.
  120. Cronin, Kurian & Snowdon 2005, p. 8.
  121. Cronin, Kurian & Snowdon 2005, p. 9.
  122. Molesti & Majolo 2016, pp. 133–135.
  123. Molesti & Majolo 2016, p. 140.
  124. Rekers, Haun & Tomasello 2011, p. 1757.
  125. Drea & Carter 2009, p. 970.
  126. Drea & Carter 2009, p. 972.
  127. Drea & Carter 2009, p. 971.
  128. Drea & Carter 2009, p. 974.
  129. Drea & Carter 2009, p. 973.
  130. Drea & Carter 2009, p. 975.
  131. Ostojić & Clayton 2014, p. 455.
  132. Ostojić & Clayton 2014, pp. 455–456.
  133. Ostojić & Clayton 2014, p. 456.
  134. Marshall-Pescini et al. 2017, p. 11796.
  135. Plotnik et al. 2011, pp. 5118–5119.
  136. Plotnik et al. 2011, p. 5119.
  137. Schmelz et al. 2017, p. 1107.
  138. Schmelz et al. 2017, p. 1108.
  139. Schmelz et al. 2017, p. 1109.
  140. Schmelz et al. 2017, p. 1110.
  141. Schmelz et al. 2017, p. 1112.
  142. Eskelinen, Winship & Jones 2016.
  143. Kuczaj, Winship & Eskelinen 2015, p. 545.
  144. Kuczaj, Winship & Eskelinen 2015, p. 546.
  145. Kuczaj, Winship & Eskelinen 2015, p. 548.
  146. King et al. 2016.
  147. Seed, Clayton & Emery 2008, pp. 1423–1424.
  148. Seed, Clayton & Emery 2008, p. 1425.
  149. Seed, Clayton & Emery 2008, p. 1427.
  150. Scheid & Noë 2010, p. 546.
  151. Scheid & Noë 2010, p. 549.
  152. Scheid & Noë 2010, p. 550.
  153. Scheid & Noë 2010, pp. 551–552.
  154. Scheid & Noë 2010, p. 552.
  155. Asakawa-Haas et al. 2016, p. 1.
  156. Massen, Ritter & Bugnyar 2015, p. 7.
  157. Asakawa-Haas et al. 2016, pp. 1, 3.
  158. Péron et al. 2011, p. 551.
  159. Péron et al. 2011, p. 552.
  160. Schwing et al. 2016, p. 1094.
  161. Heaney, Gray & Taylor 2017, p. 2.
  162. Heaney, Gray & Taylor 2017, p. 8.
  163. Heaney, Gray & Taylor 2017, pp. 6,9.
  164. Heaney, Gray & Taylor 2017, p. 9.
  165. Schwing et al. 2016, pp. 1094, 1100.
  166. Schwing et al. 2016, p. 1099.

Bibliography
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.