Falsifiability

In the philosophy of science, falsifiability or refutability is the capacity for a statement, theory or hypothesis to be contradicted by evidence. For example, the statement "All swans are white" is falsifiable because one can observe that black swans exist.[upper-alpha 1]

The observation of a black swan falsifies the hypothesis "All swans are white".

Falsifiability was introduced by the philosopher of science Karl Popper in his book Logik der Forschung (1934, revised and translated into English in 1959 as The Logic of Scientific Discovery). He proposed it as the cornerstone of a solution to both the problem of induction and the problem of demarcation.

Popper argued for falsifiability and opposed this to the intuitively similar concept of verifiability. Whereas verifying the claim "All swans are white" would require assessment of all swans, which is not possible, the single observation of a black swan is sufficient to falsify it.

As a key notion in the separation of science from non-science, falsifiability has featured prominently in many scientific controversies and applications, even being used as legal precedent.

The problem of induction

See also: Inductivism and Problem of induction

One of the questions in scientific method is: how does one move from observations to scientific laws? This is the problem of induction. Suppose we want to put the theory that all swans are white to the test. We come across a white swan. We cannot validly argue (or induce) from "here is a white swan" to "all swans are white"; doing so would require a logical fallacy such as, for example, affirming the consequent.[upper-alpha 2][1]

Popper's idea to solve this problem is that while it is impossible to verify that every swan is white, finding a single black swan shows that not every swan is white. We might tentatively accept the proposal that every swan is white, while looking out for examples of non-white swans that would show our conjecture to be false. Falsification uses the valid inference modus tollens: if from a statement ${\displaystyle P}$ (say some law with some initial condition) we logically deduce ${\displaystyle Q}$, but what is observed is ${\displaystyle \neg Q}$, we infer that ${\displaystyle P}$ is false. For example, given the statement "all swans are white" and the initial condition "there is a swan here", we can deduce "the swan here is white", but if what is observed is "the swan here is not white" (say black), then "all swans are white" is false, or it was not a swan.

Popper's response to the problem of induction is simply that induction is actually never used in science.[upper-alpha 3] Instead, laws are conjectured, tested and the results are considered together with other aspects to decide which of these laws are applied in practice.[2] In contrast, the logical empiricism movement, which included such philosophers as Moritz Schlick, Rudolf Carnap, Otto Neurath and A.J. Ayer wanted to formalize the idea that, for a law to be scientific, it must be possible to argue on the basis of observations either in favor of its truth or its falsity. There was no consensus among these philosophers about how to achieve that, but the thought expressed by Mach's dictum that "where neither confirmation nor refutation is possible, science is not concerned" was accepted as a basic precept of critical reflection about science.[3][4]

Popper said that a demarcation criterion was possible, but we have to use the logical possibility of falsifications, which is falsifiability. He cited his encounter with psychoanalysis in the 1910s. It did not matter what observation was presented, psychoanalysis could explain it. Unfortunately, the reason why it could explain everything is that it did not exclude anything also.[upper-alpha 4] For Popper, this was a failure, because it meant that it could not make any prediction. From a logical standpoint, if one finds an observation that does not contradict a law, it does not mean that the law is true. A verification has no value in itself. But, if the law makes risky predictions and these are corroborated, Popper says, there is a reason to prefer this law over another law that makes less risky predictions or no predictions at all.[upper-alpha 5][upper-alpha 6] In the definition of falsifiability, contradictions with observations are not used for actual falsifications, but for logical "falsifications" that show that the law makes risky predictions, which is completely different.

On the basic philosophical side of this issue, Popper said that some philosophers of the Vienna Circle had mixed two different problems, that of meaning and that of demarcation, and had proposed in verificationism a single solution to both: a statement that could not be verified was considered meaningless. In opposition to this view, Popper said that there are meaningful theories that are not scientific, and that, accordingly, a criterion of meaningfulness does not coincide with a criterion of demarcation.[upper-alpha 7]

The problems of falsification

Any approach that relies on actual falsifications faces two challenges. The first challenge corresponds to decisions that must be agreed upon by scientists before an attempt to falsify a theory can be successful. The second challenge is how to use falsifications (successful attempts) and corroborations (rejected attempts) to explain progress in science. Imre Lakatos said that there were two incorrect approaches, which he called dogmatic falsificationism and naive falsificationism. Dogmatic falsificationism ignores both challenges, whereas naive falsificationism addresses the first challenge only.[5] Lakatos contrasted them with sophisticated falsificationism,[6] his own improvement on Popper's solution. Popper's methodology is not (and has never been) based on one of the two incorrect approaches.[upper-alpha 8] On the terminological side of this issue, Popper said that he never referred to his methodology as "falsificationism",[upper-alpha 9] tended to avoid this term[upper-alpha 10] and proposed instead the term "critical rationalism".[upper-alpha 11]

Dogmatic falsificationism

A dogmatic falsificationist ignores that every observation is theory impregnated. This leads to the critique that it is unclear which theory is falsified. Is it the one that is being studied or the one behind the observation?[upper-alpha 12] This is sometimes called the 'Duhem–Quine problem'. An example is Galileo's refutation of the theory that celestial bodies are faultless crystal balls. Many considered that it was the optical theory of the telescope that was false, not the theory of celestial bodies. Another example is the theory that neutrinos are emitted in beta decays. Had they not been observed in the Cowan–Reines neutrino experiment, many would have considered that the strength of the beta-inverse reaction used to detect the neutrinos was not sufficiently high. At the time, Grover Maxwell wrote, the possibility that this strength was sufficiently high was a "rather pious hope".[7]

A dogmatic falsificationist ignores the role of auxiliary hypotheses, which could explain the contradicting observation. For the falsification to logically occur, a ceteris paribus clause must say that no auxiliary hypothesis is responsible for the contradicting observation. Again, this leads to the critique that it cannot be told if it is the theory or the ceteris paribus clause that is false. Lakatos gives the example of the path of a planet. If the path contradicts Newton's law, we will not know if it is Newton's law that is false or the assumption that no other body influenced the path. Popper was aware that one can always find another auxiliary hypothesis,[upper-alpha 13] though he clearly distinguished falsifiable theories such as Newton theory and non-falsifiable theories on this respect.[upper-alpha 14]

Lakatos says that Popper's solution to these criticisms requires that one relaxes the assumption that an observation can show a theory to be false:[upper-alpha 15]

If a theory is falsified [in the usual sense], it is proven false; if it is falsified [in the technical sense], it may still be true.

— Imre Lakatos, Lakatos 1978, p. 24

Methodological falsificationism replaces the contradicting observation in a falsification with a "contradicting observation" accepted by convention among scientists, a convention that implies three decisions: the theory underlying the observation is correct, no auxiliary hypotheses explain this observation and the written form of the observation matches with an actual observation.[upper-alpha 16] The falsifiers thus depend on decisions made by scientists in view of the currently accepted technology and its associated theory. So, Popper says that "Science does not rest upon solid bedrock".[upper-alpha 17] He also says (see section § Basic statements and the definition of falsifiability) that it's not an obstacle to the definition of an empirical basis and of falsifiability.

Naive falsificationism

According to Lakatos, naive falsificationism is the claim that methodological falsifications can by themselves explain how scientific knowledge progresses. Very often one must deal with two or more competing theories which are both corroborated. Considering only falsifications, it is not clear why one theory is chosen above the other, even when one is corroborated more often than the other. In fact, a stronger version of the Quine-Duhem thesis says that it's not always possible to rationally pick one theory over the other using falsifications.[8] Considering only falsifications, it is not clear why often a corroborating experiment is seen as a sign of progress. Popper's critical rationalism uses both falsifications and corroborations to explain progress in science.[upper-alpha 18] How corroborations and falsifications can explain progress in science was a subject of disagreement between many philosophers, especially between Lakatos and Popper.[upper-alpha 19]

Popper distinguished between the creative and informal process from which theories and accepted basic statements emerge and the logical and formal process where theories are falsified or corroborated.[upper-alpha 20][upper-alpha 21][upper-alpha 22] The main issue is whether the decision to select a theory among competing theories in the light of falsifications and corroborations should be moved in the logical part as some kind of formal logic.[upper-alpha 23] It is a delicate question, because this logic would be inductive: it selects a universal law in view of instances. The answer of Lakatos and many others to that question is that it should.[upper-alpha 24][upper-alpha 25] In contradistinction, for Popper, the creative and informal part is guided by methodological rules, which naturally say to favor theories that are corroborated,[upper-alpha 26] but this methodology can hardly be made rigorous.[upper-alpha 27] Popper does not discuss corroborations when he describes the purely logical part.[upper-alpha 28]

Popper's way to analyze progress in science was through the concept of verisimilitude, a way to define how close a theory is to the truth, which he did not consider very significant, except (as an attempt) to describe a concept already clear in practice. Later, it was shown that the specific definition proposed by Popper cannot distinguish between two theories that are false, which is the case for all theories in the history of science.[upper-alpha 29] Today, there is still on going research on the general concept of verisimilitude.[9]

Basic statements and the definition of falsifiability

Popper distinguished between the logic of science and its applied methodology.[upper-alpha 20] The logical part consists of theories, statements and their purely logical relationship. The methodological part consists, in Popper's view, of informal rules, which are used to guess theories, accept observation statements as factual, etc. When this distinction is applied to the term "falsifiability", it corresponds to a distinction between two completely different meanings of the term. The same is true for the term "falsifiable". Popper said that he only uses "falsifiability" or "falsifiable" in reference to the logical side and that, when he refers to the methodological side, he speaks instead of "falsification" and its problems.[upper-alpha 15]

Popper said that methodological problems require proposing methodological rules. For example, one such rule is that, if one refuses to go along with falsifications, then one has retired oneself from the game of science.[10] The logical side does not have such methodological problems, in particular with regard to the falsifiability of a theory, because basic statements are not required to be possible. Methodological rules are only needed in the context of actual falsifications.

So observations have two purposes in Popper's view. On the methodological side, observations can be used to show that a law is false, which Popper calls falsification. On the logical side, observations, which are purely logical constructions, do not show a law to be false, but contradict a law to show its falsifiability. Unlike falsifications and free from the problems of falsification, these contradictions establish the value of the law, which may eventually be corroborated. He wrote that an entire literature exists because this distinction was not understood.[upper-alpha 30]

Basic statements

In Popper's view of science, statements of observation can be constructed within a logical structure independently of any factual observations.[upper-alpha 31][11] The set of all purely logical observations that can be created constitutes the empirical basis. Popper calls them the basic statements or test statements. They are the statements that can be used to show the falsifiability of a theory. Popper says that a basic statement does not have to be possible in practice. It is sufficient that the values or properties that appear in the observations are associated with technologies.[upper-alpha 32] See the examples in section § Examples of demarcation and applications.

In more than twelve pages of The Logic of Scientific Discovery (Popper 1959, sec. 13–15, 28), Popper discusses informally which statements among those that can be constructed in the logical structure are basic statements. A logical structure uses universal classes to define laws. For example, in the law "all swans are white" the concept of swans is a universal class. It corresponds to a set of properties that every swan must have. It is not restricted to the swans that exist, existed or will exist. Informally, a basic statement is simply a statement that concerns only a finite number of specific instances in universal classes. In particular, an existential statement such as "there exists a black swan" is not a basic statement, because it is not specific about the instance. On the other hand, "this swan here is black" is a basic statement. Popper says that it is a singular existential statement or simply a singular statement. So, basic statements are singular (existential) statements.

The definition of falsifiability

Thornton says that basic statements are statements that correspond to particular "observation-reports". He then gives Popper's definition of falsifiability:

"A theory is scientific if and only if it divides the class of basic statements into the following two non-empty sub-classes: (a) the class of all those basic statements with which it is inconsistent, or which it prohibits—this is the class of its potential falsifiers (i.e., those statements which, if true, falsify the whole theory), and (b) the class of those basic statements with which it is consistent, or which it permits (i.e., those statements which, if true, corroborate it, or bear it out)."

— Thornton, Stephen, Thornton 2016, at the end of section 3

As in the case of actual falsifiers, decisions must be taken by scientists to accept a logical structure and its associated empirical basis, but these are usually part of a background knowledge that scientists have in common and, often, no discussion is even necessary.[upper-alpha 33] The first decision described by Lakatos is implicit in this agreement, but the other decisions are not needed. This agreement, if one can speak of agreement when there is not even a discussion, exists only in principle. This is where the distinction between the logical and methodological sides of science becomes important. When an actual falsifier is proposed, the technology used is considered in details and, as described in section § Dogmatic falsificationism, an actual agreement is needed. This may require using a deeper empirical basis,[upper-alpha 34] hidden within the current empirical basis, to make sure that the properties or values used in the falsifier were obtained correctly (Andersson 2016 gives some examples).

Despite the fact that the empirical basis can be shaky, more comparable to a swamp than to a solid ground, the definition that is given above is simply the formalization of a natural requirement on scientific theories, without which, Popper says, the whole logical process of science[upper-alpha 31] would not be possible.

Initial condition and prediction in falsifiers of laws

This article starts by stating that a falsifier must have parts that are separately technologically possible. Yet § The definition of falsifiability doesn't discuss multiple parts in a potential falsifier. This requires some explanation. In his analysis of the scientific nature of universal laws, Popper arrived at the conclusion that laws must "allow us to deduce, roughly speaking, more empirical singular statements than we can deduce from the initial conditions alone." A singular statement that has one part only can not contradict a universal law. It can only contradict a prediction that is obtained from the law together with another singular statement, the initial condition. A falsifier of a law has always two parts: the initial condition and the singular statement that contradicts the prediction. There was no need to require that falsifiers have two parts in the definition itself. This made the definition more general and allowed the basic statements themselves to be falsifiable.[12]

Criteria that require that a law must be predictive, just as is required by falsifiability, Popper wrote, "have been put forward as criteria of the meaningfulness of sentences (rather than as criteria of demarcation applicable to theoretical systems) again and again after the publication of [his] book, even by critics who pooh-poohed [his] criterion of falsifiability."[13]

Examples of demarcation and applications

Newton's theory

Main article: Newton's law of universal gravitation

In response to Lakatos who suggested that Newton's theory was as hard to show falsifiable as Freud's psychoanalytic theory, Popper gave the example of an apple that moves from the ground up to a branch and then starts to dance from one branch to another.[upper-alpha 35] It is clearly impossible, yet a basic statement that is a valid potential falsifier for Newton's theory, because the position of the apple at different times can be measured.

Einstein's equivalence principle

Another example of a basic statement is "The inert mass of this object is ten times larger than its gravitational mass." This is a basic statement because the inert mass and the gravitational mass can both be measured separately, even though it never happens that they are different. It is, as described by Popper, a valid falsifier for Einstein's equivalence principle.[upper-alpha 36]

Evolution

Industrial melanism

Main article: Industrial melanism

A black-bodied and white-bodied peppered moth.

An example of a basic statement in the theory of evolution is "In this industrial area, the relative fitness of the white-bodied peppered moth is high." Here "fitness" means "reproductive success over the next generation", which is the definition used as early as 1930 by Ronald Fisher[upper-alpha 37] and still the most widely accepted definition in modern biology.[upper-alpha 38] This is an example of a basic statement, because it is possible to separately determine the kind of environment, industrial vs natural, and the relative fitness of the white-bodied form (relative to the black-bodied form) in an area, even though it never happens that the white-bodied form has a high relative fitness in an industrial area. "In industrial areas, the black form of the peppered moth has higher relative fitness" is a famous example of a falsifiable statement that illustrates the effect of natural selection.[14]

Precambrian rabbit

Main article: Precambrian rabbit

A famous example of a basic statement from J.B.S. Haldane is "[These are] fossil rabbits in the Precambrian era." This is a basic statement because it is possible to find a fossil rabbit and to determine that the date of a fossil is in the Precambrian era, even though it never happens that the date of a rabbit fossil is in the Precambrian era. Despite opinions to the contrary,[15] some times wrongly attributed to Popper,[upper-alpha 39] this shows the scientific character of paleontology or the history of the evolution of life on Earth, because it contradicts the hypothesis in paleontology that all mammals existed in a much more recent era. Richard Dawkins adds that any other modern animal, such as a hippo, would suffice.[16][17][18]

Simple examples of non-falsifiable statements

Even if it is accepted that angels exist, the sentence "All angels have large wings" is not falsifiable, because though it is possible to observe the absence of large wings, no technology (independent of the presence of wings) exists to identify these angels.

A simple example of a non-basic statement is "this angel does not have large wings". It is not a basic statement, because though the absence of large wings can be observed, no technology (independent of the presence of wings[upper-alpha 40]) exists to identify angels. Even if it is accepted that angels exist, the sentence "All angels have large wings" is not falsifiable.

Another example from Popper of a non-basic statement is "This human action is altruistic." It is not a basic statement, because no accepted technology allows us to determine whether or not an action is motivated by self-interest. The theory "All human actions are egotistic, motivated by self-interest", with all its variants in behaviorism, psychoanalysis, individual psychology, utilitarianism, vulgar-marxism, religion, and sociology of knowledge, Popper wrote, is not falsifiable.[upper-alpha 41]

Omphalos hypothesis

Main article: Omphalos hypothesis

Some adherents of young-Earth creationism make an argument (called the Omphalos hypothesis after the Greek word for navel) that the world was created with the appearance of age; e.g., the sudden appearance of a mature chicken capable of laying eggs. This ad hoc hypothesis introduced into young-Earth creationism makes it non-falsifiable because it says that the time of creation (of a species) measured by the accepted technology is illusory and no accepted technology is proposed to measure the claimed "actual" time of creation. Popper says that it's fine to modify a theory by the introduction of an auxiliary hypothesis, but the new theory must at the least remain falsifiable, which is not the case here. One can also present the Omphalos hypothesis as an auxiliary hypothesis that is introduced into the accepted theory. In this view, the new theory remains falsifiable, but its falsifiability does not increase, because no additional observations are predicted. In both views, the ad hoc hypothesis, seen by itself, is not falsifiable because there is no way to measure the claimed "actual" time of creation that is proposed by this hypothesis. This is discussed in details by Dienes in the case of a variation on the Omphalos hypothesis, which, in addition, specifies that God made the creation in this way to test our faith.[19]

Useful metaphysical statements

Grover Maxwell discussed statements such as "All men are mortal".[20] This is not falsifiable, because it does not matter how old is a man, maybe it will die next year.[21] Maxwell said that this statement is nevertheless useful, because it is often corroborated. He coined the term "corroboration without demarcation". Popper's view is that it is indeed useful, but only because it is indirectly corroborated by the corroboration of the falsifiable law "All men die before the age of 150." For Popper, if no such a falsifiable law exists, then the metaphysical law is not useful, because it's not indirectly corroborated.[upper-alpha 42]

Clyde Cowan conducting the neutrino experiment (circa 1956)

Maxwell also used the example "All solids have a melting point." This is not falsifiable, because maybe the melting point will be reached at a higher temperature.[22][21] The law is falsifiable and more useful if we specify an upper bound on melting points or a way to calculate this upper bound.[upper-alpha 43] Another example from Maxwell is "All beta decays are accompanied with a neutrino emission from the same nucleus." This is also not falsifiable, because maybe the neutrino can be detected in a different manner. The law is falsifiable and much more useful from a scientific point of view, if the method to detect the neutrino is specified.[7]

Maxwell said that most scientific laws are metaphysical statements of this kind,[22] which, Popper said, need to be made more precise before they can be indirectly corroborated.[upper-alpha 42] In his critique of the falsifiability criterion, Maxwell considered the requirement for decisions in the falsification of, both, the emission of neutrinos (see § Dogmatic falsificationism) and the existence of the melting point.[7]

Another example, from the pepper moth example, is "In all areas, the white vs black trait of the pepper moth affects its fitness." This is also not falsifiable, because maybe the right environmental factor was not yet considered. When it is specified, namely, fitness in polluted industrial areas vs non polluted areas, then the law is falsifiable and it says which environmental factor should be considered to actually see an effect.[upper-alpha 44]

Natural selection

Main article: Survival of the fittest § Tautology

In the 5th and 6th editions of On the origin of species, following a suggestion of Alfred Russel Wallace, Darwin used "Survival of the fittest", an expression first coined by Herbert Spencer, as a synonym for "Natural Selection".[upper-alpha 45] Popper and others said that, if one uses the most widely accepted definition of "fitness" in modern biology (see subsection § Evolution), namely reproductive success itself, the expression "survival of the fittest" is a tautology.[upper-alpha 46][upper-alpha 47][upper-alpha 48]

In practice, as illustrated by the peppered moth example of section § Evolution, the questions asked are of the kind how specific traits affect the survival rate or fitness of a species when confronted by an environmental factor such as industrial pollution. Great Darwinist Ronald Fisher worked out mathematical theorems to help answer this kind of questions. But, for Popper and others, there is no (falsifiable) law of Natural Selection in this, because it only applies to some rare traits.[upper-alpha 49][upper-alpha 50] Instead, for Popper, the work of Fisher and others on Natural Selection is part of an important metaphysical research program.

Mathematics

See also: Mathematical proof, Scientific method § Relationship with mathematics, Mathematics § Mathematics as science, and Philosophy of mathematics § Popper's two senses of number statements

Popper said that not all non-falsifiable statements are useless in science. Mathematical statements are good examples. Like all formal sciences, mathematics is not concerned with the validity of theories based on observations in the empirical world, but rather, mathematics is occupied with the theoretical, abstract study of such topics as quantity, structure, space and change. Methods of the mathematical sciences are, however, applied in constructing and testing scientific models dealing with observable reality. Albert Einstein wrote, "One reason why mathematics enjoys special esteem, above all other sciences, is that its laws are absolutely certain and indisputable, while those of other sciences are to some extent debatable and in constant danger of being overthrown by newly discovered facts."[23]

Historicism

Theories of history or politics that allegedly predict future events have a logical form that renders them neither falsifiable nor verifiable. They claim that for every historically significant event, there exists an historical or economic law that determines the way in which events proceeded. Failure to identify the law does not mean that it does not exist, yet an event that satisfies the law does not prove the general case. Evaluation of such claims is at best difficult. On this basis, Popper "fundamentally criticized historicism in the sense of any preordained prediction of history"[24] and said that neither Marxism nor psychoanalysis was science,[24] although both made such claims.

Popper made a clear distinction between the original theory of Marx and what came to be known as Marxism later on.[25] For Popper, the original theory of Marx contained genuine scientific laws. Though they could not make preordained predictions, these laws constrained how changes can occur in society. One of them was that changes in society cannot "be achieved by the use of legal or political means".[upper-alpha 51] For Popper, this was testable, and in fact falsified. "Yet instead of accepting the refutations", Popper wrote, "the followers of Marx re-interpreted both the theory and the evidence in order to make them agree. ... They thus gave a 'conventionalist twist' to the theory; and by this stratagem they destroyed its much advertised claim to scientific status."[upper-alpha 52][upper-alpha 53]

Use in courts of law

Falsifiability has been used in the McLean v. Arkansas case (in 1982),[26] the Daubert case (in 1993)[27] and other cases. A survey of 303 federal judges conducted in 1998[upper-alpha 54] found that "[P]roblems with the nonfalsifiable nature of an expert's underlying theory and difficulties with an unknown or too-large error rate were cited in less than 2% of cases."[28]

McLean v. Arkansas case

In the ruling of the McLean v. Arkansas case, Judge William Overton used falsifiability as one of the criteria to determine that "creation science" was not scientific and should not be taught in Arkansas public schools as such (it can be taught as religion). In his testimony, philosopher Michael Ruse defined the characteristics which constitute science as (see Pennock 2000, p. 5 and Ruse 2010):

• It is guided by natural law;
• It has to be explanatory by reference to natural law;
• It is testable against the empirical world;
• Its conclusions are tentative, i.e., are not necessarily the final word; and
• It is falsifiable.

In his conclusion related to this criterion Judge Overton stated that

While anybody is free to approach a scientific inquiry in any fashion they choose, they cannot properly describe the methodology as scientific, if they start with the conclusion and refuse to change it regardless of the evidence developed during the course of the investigation.

— William Overton, McLean v. Arkansas 1982, at the end of section IV. (C)

Daubert standard

Main article: Daubert standard

In several cases of the United States Supreme Court, the court described scientific methodology using the five Daubert factors, which include falsifiability.[upper-alpha 55] The Daubert result cited Popper and other philosophers of science:

Ordinarily, a key question to be answered in determining whether a theory or technique is scientific knowledge that will assist the trier of fact will be whether it can be (and has been) tested. Scientific methodology today is based on generating hypotheses and testing them to see if they can be falsified; indeed, this methodology is what distinguishes science from other fields of human inquiry. Green 645. See also C. Hempel, Philosophy of Natural Science 49 (1966) ([T]he statements constituting a scientific explanation must be capable of empirical test); K. Popper, Conjectures and Refutations: The Growth of Scientific Knowledge 37 (5th ed. 1989) ([T]he criterion of the scientific status of a theory is its falsifiability, or refutability, or testability) (emphasis deleted).

— Harry Blackmun, Daubert 1993, p. 593

David H. Kaye[upper-alpha 56] said that references to the Daubert majority opinion confused falsifiability and falsification and that "inquiring into the existence of meaningful attempts at falsification is an appropriate and crucial consideration in admissibility determinations."[upper-alpha 57]

The bucket and the searchlight

Section § Examples of demarcation and applications shows that falsifiability is useful to demarcate between scientific and non-scientific theories, but why should falsifiability be the criterion for scientific theories? The first reason that comes to mind is to support a methodological rule that can conclusively falsify these theories. However, as described in § Dogmatic falsificationism, rigorously speaking, it is not possible to falsify a theory. Moreover, as described in § Naive falsificationism, falsification plays no decisive role in the choice of a new theory. Therefore, even if it was possible to rigorously falsify a theory, the usefulness of this methodological rule and indirectly of falsifiability would remain unclear. For Popper, this application of falsifiability or methodological rule suffers from the problems of falsification because it refers to an inadequate view of science, the bucket view of science.[29] For Popper (see below), the correct application of falsifiability, i.e., his methodology, is as free from the problems of falsification as falsifiability itself, because it relies on a different view, the searchlight view of science.[30]

The bucket view of science

Bucket view: Observations enter into the bucket and turn into valid statements. Next, (not shown) inference rules generate valid laws.

Some tentative explanations for the growth of scientific knowledge are based on what Popper calls the bucket view of science. In this view, observation statements accumulate in a bucket through observations and various procedures are used to make sure that they are valid.[upper-alpha 58] Next, new laws are obtained in a way that can be justified using inference rules that are allowed to process all the knowledge that is available in the bucket.[upper-alpha 59] In this justificational picture, Hume said that we cannot obtain new universal laws (except what can be obtained through deductive rules). Hume's argument is based on reasonable premises: non-deductive rules are in need of justification, circular arguments are not valid, etc. If we accept Hume's premises, even probabilistic attempts to explain the growth of knowledge in terms of the bucket view of science, Popper stated, are doomed to fail.[31]

Popper's solution to this problem is simply to reject the bucket view of science. His main argument is basically that he accepts Hume's argument, which shows that the bucket view fails to explain the growth of objective knowledge. Popper said that the processes in the bucket are better seen as physical processes and the laws that govern these processes are biological.[upper-alpha 60] To help people get rid of the limitations associated with the bucket view, Popper brought out the main issue with this view: it ignores the organismic aspect of knowledge.[upper-alpha 61] Given that the bucket view is a dead end, it is natural to accept that biological predispositions and expectations play an important role in these processes.[upper-alpha 62]

The searchlight view of science

Searchlight view: Expectations and predispositions turn into conjectures that act like a searchlight and lead to observations (not shown).

Popper proposed to replace the bucket view of science with what he called the searchlight view of science. In that view, Popper wrote, there is no reason why any methodology should work. It is easy, Popper said, to imagine universes where no methodology can work or even only exist.[upper-alpha 63] If you want to believe that the methodology will work, it must be postulated as an axiom. In Popper's case, the axiom is that the methodology of conjectures and refutations is going to work.[upper-alpha 64] The conjectures are the searchlight, because they lead to observational results. But this axiom will not help any objective rule in the justification of scientific knowledge.[upper-alpha 65] There is no point in attempting any justification in the searchlight view. For a popperian, the absence of these objective rules is expected. It is not a failure. In this line of thought, Einstein wrote that there is no logical path to science.[upper-alpha 66][upper-alpha 67]

Popper's scientific methodology that accompanies falsifiability contains rules such as "He who decides one day that scientific statements do not call for any further test, and that they can be regarded as finally verified, retires from the game."[10] In general, the rules of Popper's methodology influence which theories will be chosen or rejected, but these rules do that only through decisions taken by the scientists.[upper-alpha 68] As described in § Methodless creativity versus inductive methodology, every rule to determine or choose theories must rely on the good judgement of the scientists.

Why should falsifiability be the criterion for scientific theories?

Back to the original question: why should falsifiability be the criterion? It is not that falsification directly leads to the rejection of a theory. That would be a rule of the bucket view of science. It is not that we must always look for theories that are more falsifiable. That would also be a rule of the bucket view of science. Popper's main methodological rule is that scientists must try to guess and corroborate (or equivalently falsify) bold and useful conjectures and take any falsification as a problem that can be used to start a critical discussion. In other words, the usefulness of falsifiability is that falsifiable conjectures say more, because they prohibit more and, in the case of their falsification, they lead to useful problems, which steer the creative process of science. For Popper, who knew most of section § Examples of demarcation and applications, this is exactly what we should expect from a scientific theory.

The original answer provided, which is the possibility to actually falsify a theory as if there was a rule to eliminate laws, missed the whole point: No rule can supersede the hidden organic aspect of knowledge in the processing of the available objective knowledge. If there is any inference rule, the final decision must still be left to the organic aspect or, more simply, to the scientists. For Popper, inference rules are tantamount to a "quest for certainty", which he saw as the main weakness of the bucket view of science.[32]

In § #Methodless creativity versus inductive methodology, it is seen that Lakatos reached the same conclusion in the following sense that he said that his methodology did not offer any "firm heuristic advice about what to do".[upper-alpha 69] Before Popper's time, in 1906, being aware of the problems of falsification, Pierre Duhem reached the same conclusion.[upper-alpha 70] Popper reemphasized non-justificationism, which was a good match for his added falsifiability criterion and associated critical methodology.

Controversies

Methodless creativity versus inductive methodology

As described in section § Naive falsificationism, Lakatos and Popper agreed that universal laws cannot be logically deduced (except from laws that say even more), but he felt that, if the explanation for new laws can not be deductive, it must be inductive. He urged Popper explicitly to adopt some inductive principle[upper-alpha 71] and sets himself the task to find an inductive methodology.[upper-alpha 72] However, the methodology that he found did not offer any exact inductive rules. In a response to Kuhn, Feyerabend and Musgrave, Lakatos acknowledged that the methodology depends on the good judgment of the scientists.[upper-alpha 69] Feyerabend wrote in "Against Method" that Lakatos' methodology of scientific research programmes is epistemological anarchism in disguise[upper-alpha 73] and Musgrave made a similar comment.[upper-alpha 74] In more recent work, Feyerabend says that Lakatos uses rules, but whether or not to follow any of these rules is left to the judgment of the scientists.[upper-alpha 75] This is also discussed elsewhere.[upper-alpha 76]

Popper also offered a methodology with rules, but these rules are also not inductive rules, because they are not by themselves used to accept laws or establish their validity. They do that through the creativity or "good judgment" of the scientists only. For Popper, the required non deductive component of science never had to be an inductive methodology. He always viewed this component as a creative process beyond the explanatory reach of any rational methodology, but yet used to decide which theories should be studied and applied, find good problems and guess useful conjectures.[upper-alpha 77] Quoting Einstein to support his view, Popper said that this renders obsolete the need for an inductive methodology or logical path to the laws. For Popper, no inductive methodology was ever proposed to satisfactorily explain science.

Ahistorical versus historiographical

Main article: Imre_Lakatos § Research_programmes

Section § Methodless creativity versus inductive methodology says that both Lakatos's and Popper's methodology are not inductive. Yet Lakatos's methodology extended importantly Popper's methodology: it added a historiographical component to it. This allowed Lakatos to find corroborations for his methodology in the history of science. The basic units in his methodology, which can be abandoned or pursued, are research programmes. Research programmes can be degenerative or progressive and only degenerative research programmes must be abandoned at some point. For Lakatos, this is mostly corroborated by facts in history.

In contradistinction, Popper did not propose his methodology as a tool to reconstruct the history of science. Yet, some times, he did refer to history to corroborate his methodology. For example, he remarked that theories that were considered great successes were also the most likely to be falsified. Zahar's view was that, with regard to corroborations found in the history of science, there was only a difference of emphasis between Popper and Lakatos.

As an anecdotal example, in one of his articles Lakatos challenged Popper to show that his theory was falsifiable: he asked "Under what conditions would you give up your demarcation criterion?".[33] Popper replied "I shall give up my theory if Professor Lakatos succeeds in showing that Newton's theory is no more falsifiable by 'observable states of affairs' than is Freud's."[34]

Normal science versus revolutionary science

Main article: Paradigm shift

Thomas Kuhn analyzed what he calls periods of normal science as well as revolutions from one period of normal science to another,[35] whereas Popper's view is that only revolutions are relevant.[upper-alpha 78][upper-alpha 79] For Popper, the role of science, mathematics and metaphysics, actually the role of any knowledge, is to solve puzzles.[upper-alpha 80] In the same line of thought, Kuhn observes that in periods of normal science the scientific theories, which represent some paradigm, are used to routinely solve puzzles and the validity of the paradigm is hardly in question. It's only when important new puzzles emerge that cannot be solved by accepted theories that a revolution might occur. This can be seen as a viewpoint on the distinction made by Popper between the informal and formal process in science (see section § Naive falsificationism). In the big picture presented by Kuhn, the routinely solved puzzles are corroborations. Falsifications or otherwise unexplained observations are unsolved puzzles. All of these are used in the informal process that generates a new kind of theory. Kuhn says that Popper emphasizes formal or logical falsifications and fails to explain how the social and informal process works.

Astrology: falsified or not falsifiable?

Main article: Astrology

Popper often uses astrology as an example of a pseudo-science. He says that it is not falsifiable because both the theory itself and its predictions are too imprecise.[upper-alpha 81] Kuhn, as an historian of science, remarked that many predictions made by astrologers in the past were quite precise and they were very often falsified. He also said that astrologers themselves acknowledged these falsifications.[upper-alpha 82]

Anything goes versus scientific method

Main article: Epistemological anarchism

Paul Feyerabend rejected any prescriptive methodology at all. He rejected Lakatos' argument for ad hoc hypothesis, arguing that science would not have progressed without making use of any and all available methods to support new theories. He rejected any reliance on a scientific method, along with any special authority for science that might derive from such a method.[36] He said that if one is keen to have a universally valid methodological rule, epistemological anarchism or anything goes would be the only candidate.[37] For Feyerabend, any special status that science might have derives from the social and physical value of the results of science rather than its method.[38]

Sokal and Bricmont

In their book Fashionable Nonsense (from 1997, published in the UK as Intellectual Impostures) the physicists Alan Sokal and Jean Bricmont criticised falsifiability.[39] They include this critique in the "Intermezzo" chapter, where they expose their own views on truth in contrast to the extreme epistemological relativism of postmodernism. Even though Popper is clearly not a relativist, Sokal and Bricmont discuss falsifiability because they see postmodernist epistemological relativism as a reaction to Popper's description of falsifiability, and more generally, to his theory of science.[40]

See also

• Black swan theory – Theory of response to surprise events
• Contingency (philosophy)
• Defeasible reasoning – Reasoning that is rationally compelling, though not deductively valid
• Fallibilism
• Metaphysical solipsism
• Methodological solipsism
• Philosophical razor – Principle or rule of thumb that allows one to eliminate unlikely explanations for a phenomenon
  • Mike Alder § Newton's flaming laser sword
  • Occam's razor – Philosophical principle of selecting the solution with the fewest assumptions
• Philosophy of mathematics
• Pragmatic maxim
• Precambrian rabbit
• Raven paradox – A paradox arising from the question of what constitutes evidence for a statement
• Russell's teapot – Analogy coined by Bertrand Russell
• Scientific method – Interplay between observation, experiment and theory in science
  • Adversarial collaboration
  • Experimentum crucis
  • Explanatory power
  • Hypothetico-deductive model
  • Models of scientific inquiry
  • Predictive power
  • Reproducibility
  • Statistical hypothesis testing – Method of statistical inference
  • Superseded scientific theories
  • Theory-ladenness
• Tautology (logic) – Logical formula which is true in every possible interpretation
• Trial and error

Notes

1. "All swans are white" is often chosen as an example of a falsifiable statement, because for some 1500 years, the black swan existed in the European imagination as a metaphor for that which could not exist. Had the presumption concerning black swans in this metaphor be right, the statement would still have been falsifiable.
2. The valid inference rule modus tollens says that, given the implication p ⟶ q, if the antecedent p is affirmed, we can infer the consequent q. The fallacy "affirming the consequent" flips the antecedent with the consequent and says that, given the same implication, if the consequent q is affirmed, we can infer the antecedent p. This fallacy supports an argument from the observed q="here is a white swan" to p'="all swans are white", because, introducing p="all swans are white and here is a swan", we have p ⟶ q and p ⟶ p' and thus, if the consequent q is affirmed, we can (fallaciously) infer p and then legitimately infer p'.
3. Popper 1983, chap. 1, sec. 3: "It seems that almost everybody believes in induction; believes, that is, that we learn by the repetition of observations. Even Hume, in spite of his great discovery that a natural law can neither be established nor made 'probable' by induction, continued to believe firmly that animals and men do learn through repetition: through repeated observations as well as through the formation of habits, or the strengthening of habits, by repetition. And he upheld the theory that induction, though rationally indefensible and resulting in nothing better than unreasoned belief, was nevertheless reliable in the main— more reliable and useful at any rate than reason and the processes of reasoning; and that 'experience' was thus the unreasoned result of a (more or less passive) accumulation of observations. As against all this, I happen to believe that in fact we never draw inductive inferences, or make use of what are now called 'inductive procedures'. Rather, we always discover regularities by the essentially different method of trial and error."
4. Popper 1963, p. 35: "As for Adler, I was much impressed by a personal experience. Once, in 1919, I reported to him a case which to me did not seem particularly Adlerian, but which he found no difficulty in analysing in terms of his theory of inferiority feelings, although he had not even seen the child. Slightly shocked, I asked him how he could be so sure. 'Because of my thousandfold experience,' he replied; whereupon I could not help saying: 'And with this new case, I suppose, your experience has become thousand-and-one-fold.'"
5. Thornton 2007, p. 3: "However, a theory that has successfully withstood critical testing is thereby 'corroborated', and may be regarded as being preferable to falsified rivals. In the case of rival non-falsified theories, for Popper, the higher the informative content of a theory the better it is scientifically, because every gain in content brings with it a commensurate gain in predictive scope and testability."
6. Popper 1959, p. 19: "Various objections might be raised against the criterion of demarcation here proposed. In the first place, it may well seem somewhat wrong-headed to suggest that science, which is supposed to give us positive information, should be characterized as satisfying a negative requirement such as refutability. However, I shall show, in sections 31 to 46, that this objection has little weight, since the amount of positive information about the world which is conveyed by a scientific statement is the greater the more likely it is to clash, because of its logical character, with possible singular statements. (Not for nothing do we call the laws of nature 'laws': the more they prohibit the more they say.)"
7. Feigl 1978: "Karl Popper, an Austrian-born British philosopher of science, in his Logik der Forschung (1935; The Logic of Scientific Discovery), insisted that the meaning criterion should be abandoned and replaced by a criterion of demarcation between empirical (scientific) and transempirical (nonscientific, metaphysical) questions and answers—a criterion that, according to Popper, is to be testability."
8. Lakatos refers to Popper 1959 when he credits Popper for sophisticated falsificationism. See Lakatos 1978, pages 33, 35 and the note at page 143.
9. Popper 1983, Introduction, IV: "This may be the place to mention, and to refute, the legend that Thomas S. Kuhn, in his capacity as a historian of science, is the one who has shown that my views on science (sometimes, but not by me, called 'falsificationismism') can be refuted by the facts; that is to say, by the history of science."
10. Popper 1978, Introduction, 1978: "In connection with the term "falsificationism" (which now I tend to avoid), I would like to note that I have never said that falsification is important, or that it is more important than verification. ... The only verifications of significance are serious attempts at falsification that have not achieved their objective, thus resulting in a verification rather than a falsification."
11. Popper 1963, p. 26: "The proper answer to my question 'How can we hope to detect and eliminate error?' is, I believe, 'By criticizing the theories or guesses of others and—if we can train ourselves to do so—by criticizing our own theories or guesses.' (The latter point is highly desirable, but not indispensable; for if we fail to criticize our own theories, there may be others to do it for us.) This answer sums up a position which I propose to call 'critical rationalism'."
12. Popper 1963, p. 111: "Against the view here developed one might be tempted to object (following Duhem 28 ) that in every test it is not only the theory under investigation which is involved, but also the whole system of our theories and assumptions—in fact, more or less the whole of our knowledge—so that we can never be certain which of all these assumptions is refuted. But this criticism overlooks the fact that if we take each of the two theories (between which the crucial experiment is to decide) together with all this background knowledge, as indeed we must, then we decide between two systems which differ only over the two theories which are at stake. It further overlooks the fact that we do not assert the refutation of the theory as such, but of the theory together with that background knowledge; parts of which, if other crucial experiments can be designed, may indeed one day be rejected as responsible for the failure. (Thus we may even characterize a theory under investigation as that part of a vast system for which we have, if vaguely, an alternative in mind, and for which we try to design crucial tests.)"
13. Popper 1959, p. 19: "It might be said that even if the asymmetry [between universal and existential statements] is admitted, it is still impossible, for various reasons, that any theoretical system should ever be conclusively falsified. For it is always possible to find some way of evading falsification, for example by introducing ad hoc an auxiliary hypothesis, or by changing ad hoc a definition. It is even possible without logical inconsistency to adopt the position of simply refusing to acknowledge any falsifying experience whatsoever. Admittedly, scientists do not usually proceed in this way, but logically such procedure is possible; and this fact, it might be claimed, makes the logical value of my proposed criterion of demarcation dubious, to say the least."
14. Lakatos says that, if the ceteris paribus clause is false, both Newton theory and Freud theory can avoid a logical falsification, but he fails to say that for many falsifiers of Newton theory, the clause is true under normal assumptions and thus is not really required as a separate clause. Popper was concerned that Lakatos meant that Newton theory could be put in the same category as Freud theory and wrote a response of 5 pages in Popper 1974, pp. 1004–1009 to discuss this issue. See also Popper 2009, Introduction, 1978: "[S]ome of my former students ... believe that any putative falsification of Newtonian theory may be turned into a victory by assuming the existence of an unknown (and perhaps invisible) mass. However, this is simply a physical (or mathematical) error. First, there are many motions that in principle are observable but that cannot be explained by any such auxiliary hypothesis (for instance, a sudden reversal of motions)."
15. Popper 1983, p. XXII: "We must distinguish two meanings of the expressions falsifiable and falsifiability:
    "1) Falsifiable as a logical-technical term, in the sense of the demarcation criterion of falsifiability. This purely logical concept — falsifiable in principle, one might say — rests on a logical relation between the theory in question and the class of basic statements (or the potential falsifiers described by them).
    "2) Falsifiable in the sense that the theory in question can definitively or conclusively or demonstrably be falsified ("demonstrably falsifiable").
    "I have always stressed that even a theory which is obviously falsifiable in the first sense is never falsifiable in this second sense. (For this reason I have used the expression falsifiable as a rule only in the first, technical sense. In the second sense, I have as a rule spoken not of falsifiability but rather of falsification and of its problems)"
    
16. Lakatos add two other decisions. One of them is needed to accept statistical statements as falsifiable. The other allows even more falsifiable theories.
17. Popper 1959, p. 94: "Science does not rest upon solid bedrock. The bold structure of its theories rises, as it were, above a swamp. It is like a building erected on piles. The piles are driven down from above into the swamp, but not down to any natural or 'given' base; and if we stop driving the piles deeper, it is not because we have reached firm ground. We simply stop when we are satisfied that the piles are firm enough to carry the structure, at least for the time being."
18. Popper 1959, p. 91: "It may now be possible for us to answer the question: How and why do we accept one theory in preference to others? The preference is certainly not due to anything like a experiential justification of the statements composing the theory; it is not due to a logical reduction of the theory to experience. We choose the theory which best holds its own in competition with other theories; the one which, by natural selection, proves itself the fittest to survive. This will be the one which not only has hitherto stood up to the severest tests, but the one which is also testable in the most rigorous way. A theory is a tool which we test by applying it, and which we judge as to its fitness by the results of its applications."
19. Lakatos says that Popper is not the sophisticated falsificationist that he describes, but not the naive falsificationist either (see Lakatos 1978): "In an earlier paper,' I distinguished three Poppers: Popper0, Popper1, and Popper2. Popper0 is the dogmatic falsificationist ... Popper1 is the naive falsificationist, Popper2 the sophisticated falsificationist. ... The real Popper has never explained in detail the appeal procedure by which some 'accepted basic statements', may be eliminated. Thus the real Popper consists of Popper1 together with some elements of Popper2."
20. Thornton 2016, sec. 3: "Popper has always drawn a clear distinction between the logic of falsifiability and its applied methodology. The logic of his theory is utterly simple: if a single ferrous metal is unaffected by a magnetic field it cannot be the case that all ferrous metals are affected by magnetic fields. Logically speaking, a scientific law is conclusively falsifiable although it is not conclusively verifiable. Methodologically, however, the situation is much more complex: no observation is free from the possibility of error—consequently we may question whether our experimental result was what it appeared to be."
21. Popper clearly distinguishes between the methodological rules and the rules of pure logic (see Popper 1959, p. 32): "Methodological rules are here regarded as conventions. They might be described as the rules of the game of empirical science. They differ from the rules of pure logic..."
22. Popper 1959, p. 27: "The theory of method, in so far as it goes beyond the purely logical analysis of the relations between scientific statements, is concerned with the choice of methods—with decisions about the way in which scientific statements are to be dealt with."
23. Zahar wrote a brief summary of Lakatos's position regarding Popper's philosophy. He says (see Zahar 1983, p. 149): "The important question of the possibility of a genuine logic of [scientific] discovery" is the main divergence between Lakatos and Popper. About Popper's view, Zahar wrote (see Zahar 1983, p. 169): "To repeat: Popper offers a Darwinian account of the progress of knowledge. Progress is supposed to result negatively from the elimination by natural selection of defective alternatives. ... There is no genuine logic of discovery, only a psychology of invention juxtaposed to a methodology which appraises fully fledged theories."
24. In Lakatos terminology, the term "falsified" has a different meaning for a naive falsificationist than for a sophisticated falsificationist. Putting aside this confusing terminological aspect, the key point is that Lakatos wanted a formal logical procedure to determine which theories we must keep (see Lakatos 1978, p. 32): "For the naive falsificationist a theory is falsified by a ('fortified') 'observational' statement which conflicts with it (or which he decides to interpret as conflicting with it). For the sophisticated falsificationist a scientific theory T is falsified if and only if another theory T' has been proposed with the following characteristics: ( 1 ) T' has excess empirical content over T: that is, it predicts novel facts, that is, facts improbable in the light of, or even forbidden, by (2) T' explains the previous success of T, that is, all the unrefuted content of T is included (within the limits of observational error) in the content of T'; and (3) some of the excess content of T' is corroborated."
25. In his critique of Popper (see Kuhn 1965, p. 15), Kuhn says that the methodological rules are not sufficient to provide a logic of discovery: "... rules or conventions like the following: 'Once a hypothesis has been proposed and tested, and has proved its mettle, it may not be allowed to drop out without 'good reason'. A 'good reason' may be, for instance: replacement of the hypothesis by another which is better testable; or the falsification of one of the consequences of the hypothesis.'
    Rules like these, and with them the entire logical enterprise described above, are no longer simply syntactic in their import. They require that both the epistemological investigator and the research scientist be able to relate sentences derived from a theory not to other sentences but to actual observations and experiments. This is the context in which Sir Karl's term 'falsification' must function, and Sir Karl is entirely silent about how it can do so."
26. Popper gives an example of a methodological rule that uses corroborations (see Popper 1959, p. 32): "Once a hypothesis has been proposed and tested, and has proved its mettle, it may not be allowed to drop out without 'good reason'. A 'good reason' may be, for instance: replacement of the hypothesis by another which is better testable; or the falsification of one of the consequences of the hypothesis."
27. Popper 1959, section 23, 1st paragraph: "The requirement of falsifiability which was a little vague to start with has now been split into two parts. The first, the methodological postulate (cf. section 20), can hardly be made quite precise. The second, the logical criterion, is quite definite as soon as it is clear which statements are to be called 'basic'."
28. Popper 1959, p. 9: "According to the view that will be put forward here, the method of critically testing theories, and selecting them according to the results of tests, always proceeds on the following lines. From a new idea, put up tentatively, and not yet justified in any way—an anticipation, a hypothesis, a theoretical system, or what you will—conclusions are drawn by means of logical deduction. These conclusions are then compared with one another and with other relevant statements, so as to find what logical relations (such as equivalence, derivability, compatibility, or incompatibility) exist between them."
29. Popper 1983, Introduction, V: "The hope further to strengthen this theory of the aims of science by the definition of verisimilitude in terms of truth and of content was, unfortunately, vain. But the widely held view that scrapping this definition weakens my theory is completely baseless."
30. Some of the observations used to contradict laws might be unquestionably impossible, because the criterion must be able to accept unquestionably true laws. Besides this problem, which is resolved in the previous note, Popper describes more important problems created when we do not distinguish between falsifiability in the logical sense and falsifiability as understood in the common language (see Popper 1983, Introduction, 1982): "Although the first sense refers to the logical possibility of a falsification in principle, the second sense refers to a conclusive practical experimental proof of falsity. But anything like conclusive proof to settle an empirical question does not exist. An entire literature rests on the failure to observe this distinction." For a discussion related to this lack of distinction, see Rosende 2009, p. 142.
31. In Popper's description of the scientific procedure of testing, there is no discussion of factual observations except in those tests that compare the theory with factual observations, but in these tests too the procedure is mostly logical and involves observations that are only logical constructions (Popper 1959, pp. 9–10): "We may if we like distinguish four different lines along which the testing of a theory could be carried out. First there is the logical comparison of the conclusions among themselves, by which the internal consistency of the system is tested. Secondly, there is the investigation of the logical form of the theory, with the object of determining whether it has the character of an empirical or scientific theory, or whether it is, for example, tautological. Thirdly, there is the comparison with other theories, chiefly with the aim of determining whether the theory would constitute a scientific advance should it survive our various tests. And finally, there is the testing of the theory by way of empirical applications of the conclusions which can be derived from it. ... Here too the procedure of testing turns out to be deductive. With the help of other statements, previously accepted, certain singular statements—which we may call 'predictions'—are deduced from the theory; especially predictions that are easily testable or applicable. From among these statements, those are selected which are not derivable from the current theory, and more especially those which the current theory contradicts."
32. In practice, technologies change. When the interpretation of a theory is modified by an improved technological interpretation of some properties, the new theory can be seen as the same theory with an enlarged scope. For example, Herbert Keuth, (Keuth 2005, p. 43) wrote: "But Popper's falsifiability or testability criterion does not presuppose that a definite distinction between testable and non testable statement is possible ... technology changes. Thus a hypotheses that was first untestable may become testable later on."
33. Popper 1959, section 7, page 21: "If falsifiability is to be at all applicable as a criterion of demarcation, then singular statements must be available which can serve as premisses in falsifying inferences. Our criterion therefore appears only to shift the problem—to lead us back from the question of the empirical character of theories to the question of the empirical character of singular statements.
    "Yet even so, something has been gained. For in the practice of scientific research, demarcation is sometimes of immediate urgency in connection with theoretical systems, whereas in connection with singular statements, doubt as to their empirical character rarely arises. It is true that errors of observation occur and that they give rise to false singular statements, but the scientist scarcely ever has occasion to describe a singular statement as non-empirical or metaphysical."
34. Popper 1963, p. 387: "Before using the terms 'basic' and 'basic statement', I made use of the term 'empirical basis', meaning by it the class of all those statements which may function as tests of empirical theories (that is, as potential falsifiers). In introducing the term 'empirical basis' my intention was, partly, to give an ironical emphasis to my thesis that the empirical basis of our theories is far from firm; that it should be compared to a swamp rather than to solid ground."
35. Popper 1974, p. 1005: "... would contradict Newton's theory. This theory would equally be contradicted if the apples from one of my, or Newton's, apple trees were to rise from the ground (without there being a whirling about), and begin to dance round the branches of the apple tree from which they had fallen."
36. Popper put as an example of falsifiable statement with failed falsifications Einstein's equivalence principle. See Popper 1983, Introduction, xx: "Einstein's principle of proportionality of inert and (passively) heavy mass. This equivalence principle conflicts with many potential falsifiers: events whose observation is logically possible. Yet despite all attempts (the experiments by Eötvös, more recently refined by Rickle) to realize such a falsification experimentally, the experiments have so far corroborated the principle of equivalence."
37. Fisher 1930, p. 34: "Since m measures fitness to survive by the objective fact of representation in future generations,..."
38. For example, see Cruzan 2018, p. 156, Muehlenbein 2010, p. 21 or Ridley 2003, website complement
39. Popper 1980, p. 611: "It does appear that some people think that I denied scientific character to the historical sciences, such as palaeontology, or the history of the evolution of life on Earth. This is a mistake, and I here wish to affirm that these and other historical sciences have in my opinion scientific character; their hypotheses can in many cases be tested."
40. If the criteria to identify an angel was simply to observe large wings, then "this angel does not have large wings" would be a logical contradiction and thus not a basic statement anyway.
41. Popper 1983, Introduction, xx: "This theory ['All human actions are egotistic, motivated by self-interest'] is widely held: it has variants in behaviourism, psychoanalysis, individual psychology, utilitarianism, vulgar-marxism, religion, and sociology of knowledge. Clearly this theory, with all its variants, is not falsifiable: no example of an altruistic action can refute the view that there was an egotistic motive hidden behind it."
42. Popper 1974, p. 1038: "[A]s indeed is the case in Maxwell's example, when existential statements are verified this is done by means of stronger falsifiable statements. ... What this means is this. Whenever a pure existential statement, by being empirically "confirmed", appears to belong to empirical science, it will in fact do so not on its own account, but by virtue of being a consequence of a corroborated falsifiable theory."
43. Keuth 2005, p. 46: "[T]he existential quantifier in the symbolized version of "Every solid has a melting point" is not inevitable; rather this statement is actually a negligent phrasing of what we really mean."
44. Urban 2016, p. 22: "The concepts and relations that theories [such as Natural selection] employ exist on such an abstract level that it is often difficult to evaluate them against actual observations. Theories are therefore examined by testing hypotheses derived from these larger conceptual structures. ... These falsifiable propositions ... apply the abstract concepts of high-level theories. Take, for example, the case of northern England’s peppered moths, Biston betaluria sp. ... Biologists proposed the hypothesis that the shifting colors were a specific example of general evolutionary principles operating within specific [polluted versus non-polluted] environmental circumstances."
45. Darwin 1869, pp. 72: "I have called this principle, by which each slight variation, if useful, is preserved, by the term natural selection, in order to mark its relation to man's power of selection. But the expression often used by Mr. Herbert Spencer, of the Survival of the Fittest, is more accurate, and is sometimes equally convenient."
46. Thompson 1981, pp. 52–53, Introduction: "For several years, evolutionary theory has been under attack from critics who argue that the theory is basically a tautology. The tautology is said to arise from the fact that evolutionary biologists have no widely accepted way to independently define 'survival' and 'fitness.' That the statement, 'the fit survive,' is tautological is important, because if the critics are correct in their analysis, the tautology renders meaningless much of contemporary evolutionary theorizing. ... The definition of key evolutionary concepts in terms of natural selection runs the risk of making evolutionary theory a self-contained, logical system which is isolated from the empirical world. No meaningful empirical prediction can be made from one side to the other side of these definitions. One cannot usefully predict that nature selects the fittest organism since the fittest organism is by definition that which nature selects."
47. Waddington 1959, pp. 383–384: "Darwin's major contribution was, of course, the suggestion that evolution can be explained by the natural selection of random variations. Natural selection, which was at first considered as though it were a hypothesis that was in need of experimental or observational confirmation, turns out on closer inspection to be a tautology, a statement of an inevitable, although previously unrecognized, relation. It states that the fittest individuals in a population (defined as those which leave most offspring) will leave most offspring. Once the statement is made, its truth is apparent. This fact in no way reduces the magnitude of Darwin's achievement; only after it was clearly formulated, could biologists realize the enormous power of the principle as a weapon of explanation."
48. Popper 1994, p. 90: "If, more especially, we accept that statistical definition of fitness which defines fitness by actual survival, then the theory of the survival of the fittest becomes tautological, and irrefutable."
49. Thompson 1981, p. 53, Introduction: "Even if it did not make a tautology of evolution theory, the use of natural selection as a descriptive concept would have serious drawbacks. While it is mathematically tractable and easy to model in the laboratory, the concept is difficult to operationalize in the field. For field biologists, it is really a hypothetical entity. Clear, unambiguous instances of the operation of natural selection are difficult to come by and always greeted with great enthusiasm by biologists (Kettlewell, 1959 [the case of the peppered moths]; Shepherd, 1960). Thus, although the concept has much to recommend it as an explanatory one, it seems an overly abstract formulation on which to base a descriptive science."
50. Popper 1978, p. 342: "However, Darwin's own most important contribution to the theory of evolution, his theory of natural selection, is difficult to test. There are some tests, even some experimental tests; and in some cases, such as the famous phenomenon known as "industrial melanism", we can observe natural selection happening under our very eyes, as it were. Nevertheless, really severe tests of the theory of natural selection are hard to come by, much more so than tests of otherwise comparable theories in physics or chemistry."
51. Popper 1947, Chap.15 sec. III p. 101 harvnb error: no target: CITEREFPopper1947 (help): "In Marx’s view, it is vain to expect that any important change can be achieved by the use of legal or political means ; a political revolution can only lead to one set of rulers giving way to another set—a mere exchange of the persons who act as rulers. Only the evolution of the underlying essence, the economic reality can produce any essential or real change—a social revolution."
52. Popper 1963, p. 37: "In some of its earlier formulations (for example in Marx's analysis of the character of the 'coming social revolution') their predictions were testable, and in fact falsified. Yet instead of accepting the refutations the followers of Marx re-interpreted both the theory and the evidence in order to make them agree. In this way they rescued the theory from refutation; but they did so at the price of adopting a device which made it irrefutable. They thus gave a 'conventionalist twist' to the theory; and by this stratagem they destroyed its much advertised claim to scientific status."
53. Thornton 2016, Sec. 2: "The Marxist account of history too, Popper held, is not scientific, although it differs in certain crucial respects from psychoanalysis. For Marxism, Popper believed, had been initially scientific, in that Marx had postulated a theory which was genuinely predictive. However, when these predictions were not in fact borne out, the theory was saved from falsification by the addition of ad hoc hypotheses which made it compatible with the facts. By this means, Popper asserted, a theory which was initially genuinely scientific degenerated into pseudo-scientific dogma."
54. Surveys were mailed to all active U.S. district court judges in November 1998 (N = 619). 303 usable surveys were obtained for a response rate of 51%. See Krafka 2002, p. 9 in archived pdf.
55. The Daubert case and subsequent cases that used it as a reference, including General Electric Co. v. Joiner and Kumho Tire Co. v. Carmichael, resulted in an amendment of the Federal Rules of Evidence (see Rules of Evidence 2017, p. 15, Rule 702 and Rule 702 Notes 2011). The Kumho Tire Co. v. Carmichael case and other cases considered the original Daubert factors, but the amended rule, rule 702, even though it is often referred to as the Daubert standard, does not include the original Daubert factors or mention falsifiability or testability and neither does the majority opinion delivered by William Rehnquist in the General Electric Co. v. Joiner case.
56. Not to be confused with David Kaye (law professor), United Nations special rapporteur. David H. Kaye is distinguished professor of law at Penn State Law.
57. Kaye 2005, p. 2: "... several courts have treated the abstract possibility of falsification as sufficient to satisfy this aspect of the screening of scientific evidence. This essay challenges these views. It first explains the distinct meanings of falsification and falsifiability. It then argues that while the Court did not embrace the views of any specific philosopher of science, inquiring into the existence of meaningful attempts at falsification is an appropriate and crucial consideration in admissibility determinations. Consequently, it concludes that recent opinions substituting mere falsifiability for actual empirical testing are misconstruing and misapplying Daubert."
58. Popper says that the bucket view is a subjectivist view, but he also says that the main problem with this view is that it accepts the objective standard for growth of knowledge. For example, he wrote (Popper 1972, p. 76): "[the bucket view] is not very thoroughgoing in its subjectivism; on the contrary, the idea of ‘sufficient reason’ was originally no doubt an objectivist idea: This, I think, indicates the greatest weakness of the [bucket view] of knowledge. ... [The bucket view] accepts, consciously or unconsciously, objective demonstrable knowledge as the paradigm for all knowledge, since it is really only here that we have fully ‘sufficient reasons’ for distinguishing ‘true and certain knowledge’ from ‘mere opinion’ or ‘mere belief." Popper used a diagram of a bucket to represent the subjectivist view. In contradistinction, here, to bring out the problem that Popper described, it is the objective aspect of the bucket view that the diagram represents.
59. Popper 1972, p. 75: Popper wrote that the main weakness of the bucket view is the requirement that "the believer should be in possession of sufficient reasons for establishing that the item of knowledge is true with certainty."
60. Popper 1972, p. 71: "Knowledge in its various subjective forms is dispositional and expectational. It consists of dispositions of organisms, and these dispositions are the most important aspect of the organization of an organism."
61. Using an emphasis on a subjectivist view (see previous note), Popper said that in Hume's bucket view of science the bucket is self-contained except for observational inputs (see Popper 1972, p. 87): "To show that [Hume] thought so, I quote, from an immensity of passages, the following from the Treatise in which he argues against the ‘error’ of our belief in an external world: 'From all this it may be infer’d, that no other faculty is requir'd, beside the senses, to convince us of the external existence of body. ...' This is the bucket theory in purity: our knowledge consists of our perceptions or ‘impressions’ which ‘enter by the senses’. And these, once they constitute knowledge, must be in us, and there can be no distance or externality." It must be reminded that, for Popper, the main issue with the bucket view is the objective standard of growth that is attached to it. In this perspective, the conclusion that the bucket is self-contained means that the objective growth of knowledge has only observations as resources, no other "external" aspects, organismic or whatever.
62. Popper 1972, Appendix I, p. 343: "Before elaborating [the searchlight view of science] any further I shall try to introduce, as a digression, a few remarks of a biological nature. Although these are not meant to constitute a basis or even an argument for the main thesis which I intend to propose later, they may perhaps be helpful in getting over, or in circumventing, certain objections to it, and in this way facilitate later its understanding. ... We can say that every organism possesses a certain innate set of possible reactions, or a certain disposition to react in this or that way."
63. Popper 1972, p. 28: "[Traditional treatments of induction] all assume not only that our quest for [scientific] knowledge has been successful, but also that we should be able to explain why it is successful. However, even on the assumption (which I share) that our quest for knowledge has been very successful so far, and that we now know something of our universe, this success becomes [i. e., remains] miraculously improbable, and therefore inexplicable; for an appeal to an endless series of improbable accidents is not an explanation. (The best we can do, I suppose, is to investigate the almost incredible evolutionary history of these accidents ...)"
64. Popper 1994b, p. 498: "Though truth is not self-revealing (as Cartesians and Baconians thought), though certainty may be unattainable, the human situation with respect to knowledge is far from desperate. On the contrary, it is exhilarating: here we are, with the immensely difficult task before us of getting to know the beautiful world we live in, and ourselves; and fallible though we are we nevertheless find that our powers of understanding, surprisingly, are almost adequate for the task—more so than we ever dreamt in our wildest dreams."
65. The axiom allows to infer that the scientific knowledge that will be obtained with the methodology is going to be valid in whatever sense is attributed to 'the methodology is going to work'. One could say that the axiom is a kind of Kant's a priori knowledge, with the important distinction that the axiom is about the process of investigation by scientists, whereas Kant's a priori knowledge was about the investigated world: it involved the nature of space-time, etc. Kant's a priori knowledge has to be somehow used within the process. In contradistinction, the axiom is about the process.
66. Einstein wrote (see Yehuda 2018, p. 41 harvnb error: no target: CITEREFYehuda2018 (help)): "The supreme task of the physicist is to arrive at those universal elementary laws from which the cosmos can be built up by pure deduction. There is no logical path to these laws; only intuition, resting on sympathetic understanding of experience, can reach them."
67. Einstein wrote (see Feldman & Williams 2007, p. 151 and ): "I am convinced that we can discover by means of purely mathematical constructions the concepts and laws connecting them with each other, which furnish the key to the understanding of natural phenomena. ... Experience remains, of course, the sole criterion of the physical utility of a mathematical construction. But the creative principle resides in mathematics. In a certain sense, therefore, I hold it true that pure thought can grasp reality, as the ancients dreamed."
68. Popper 1959, p. 32: "Clearly [methodological rules] are very different from the rules usually called ‘logical’. Although logic may perhaps set up criteria for deciding whether a statement is testable, it certainly is not concerned with the question whether anyone exerts himself to test it."
69. Lakatos 1978, pp. 116–117: "The methodology of research programmes was criticized both by Feyerabend and by Kuhn. According to Kuhn: '[Lakatos] must specify criteria which can be used at the time to distinguish a degenerative from a progressive research programme; and so on. Otherwise, he has told us nothing at all.' Actually, I do specify such criteria. But Kuhn probably meant that '[my] standards have practical force only if they are combined with a time limit (what looks like a degenerating problem shift may be the beginning of a much longer period of advance)'. Since I specify no such time limit, Feyerabend concludes that my standards are no more than 'verbal ornament'. A related point was made by Musgrave in a letter containing some major constructive criticisms of an earlier draft, in which he demanded that I specify, for instance, at what point dogmatic adherence to a programme ought to be explained 'externally' rather than 'internally'. Let me try to explain why such objections are beside the point. One may rationally stick to a degenerating programme until it is overtaken by a rival and even after. What one must not do is to deny its poor public record. Both Feyerabend and Kuhn conflate methodological appraisal of a programme with firm heuristic advice about what to do. It is perfectly rational to play a risky game: what is irrational is to deceive oneself about the risk. This does not mean as much licence as might appear for those who stick to a degenerating programme. For they can do this mostly only in private."
70. In 1906, Pierre Duhem wrote in French (Duhem 1906, p. 356, translation in English in Duhem 1991, p. 216): "When certain consequences of a theory are struck by experimental contradiction, we learn that this theory should be modified but we are not told by the experiment what must be changed. It leaves to the physicist the task of finding out the weak spot that impairs the whole system. No absolute principle directs this inquiry, which different physicists may conduct in very different ways without having the right to accuse one another of illogicality. ... Pure logic is not the only rule for our judgments; certain opinions which do not fall under the hammer of the principle of contradiction are in any case perfectly unreasonable. ... Now, it may be good sense that permits us to decide between two physicists. ... But these reasons of good sense do not impose themselves with the same implacable rigor that the prescriptions of logic do."
71. Zahar 1983, p. 167: "Lakatos urged Popper explicitly to adopt some inductive principle which would synthetically link verisimilitude to corroboration."
72. Lakatos 1978, p. 112: "It should be pointed out, however, that the methodology of scientific research programmes has more teeth than Duhem's conventionalism: instead of leaving it to Duhem's unarticulated common sense to judge when a 'framework' is to be abandoned, I inject some hard Popperian elements into the appraisal of whether a programme progresses or degenerates or of whether one is overtaking another. That is, I give criteria of progress and stagnation within a programme and also rules for the 'elimination' of whole research programmes."
73. Watkins 1989, p. 6 harvnb error: no target: CITEREFWatkins1989 (help) : "Although Paul Feyerabend and Alan Musgrave evaluated [Lakatos' view] in opposite ways, they agreed about its nature. Feyerabend hailed it as an 'anarchism in disguise' (Feyerabend, Against Method, 1975), while Musgrave rather deplored the fact that Lakatos had 'gone a long way towards epistemological anarchism' (Musgrave 1976, p. 458). Musgrave added: 'Lakatos deprived his standards of practical force, and adopted a position of "anything goes"' (Musgrave 1976, p. 478)."
74. Musgrave 1976, p. 458 : "My third criticism concerns the question of whether Lakatos’s methodology is in fact a methodology in the old-fashioned sense: whether, that is, it issues in advice to scientists. I shall argue that Lakatos once had sound views on this matter, but was led, mistakenly in my opinion, to renounce them. In renouncing them, he has gone a long way towards epistemological anarchism."
75. Feyerabend 1978, p. 15: "Lakatos realized and admitted that the existing standards of rationality, standards of logic included, are too restrictive and would have hindered science had they been applied with determination. He therefore permitted the scientist to violate them... However, he demanded that research programmes show certain features in the long run — they must be progressive. In Chapter 16 of [Against Method] (and in my essay 'On the Critique of Scientific Reason': Feyerabend 1978b, p. 120) I have argued that this demand no longer restricts scientific practice. Any development agrees with it. The demand (standard) is rational, but it is also empty. Rationalism and the demands of reason have become purely verbal in the theory of Lakatos." See also Feyerabend 1981, p. 148.
76. Couvalis 1997, pp. 74-75: "There is a sense in which Feyerabend is right. Lakatos fails to give precise mechanical rules for when a theory has been finally falsified. Yet an appropriate question might be whether such rules are possible or necessary to make science rational. ... There are, however, many rough and ready rules, the application of which has to be learned in practical contexts. ... This does not mean that precise rules cannot be used in certain contexts, but we need to use our judgement to decide when those rules are to be used."
77. Zahar (Zahar 1983, p. 168) recognizes that formal rules in a methodology cannot be rational. Yet, at the level of the technology, that is, at the practical level, he says, scientists must nevertheless take decisions. Popper's methodology does not specify formal rules, but non-rational decisions will still have to be taken. He concludes that "Popper and Lakatos differ only over the levels at which they locate non-rationality in science: Lakatos at the level of an inductive principle which justifies technology, and Popper at the lower-level of technology itself."
78. Kuhn 1974, p. 802: "I suggest then that Sir Karl has characterized the entire scientific enterprise in terms that apply only to its occasional revolutionary parts. His emphasis is natural and common: the exploits of a Copernicus or Einstein make better reading than those of a Brahe or Lorentz; Sir Karl would not be the first if he mistook what I call normal science for an intrinsically uninteresting enterprise. Nevertheless, neither science nor the development of knowledge is likely to be understood if research is viewed exclusively through the revolutions it occasionally produces."
79. Watkins 1970, p. 28: "Thus we have the following clash: the condition which Kuhn regards as the normal and proper condition of science is a condition which, if it actually obtained, Popper would regard as unscientific, a state of affairs in which critical science had contracted into defensive metaphysics. Popper has suggested that the motto of science should be: Revolution in permanence! For Kuhn, it seems, a more appropriate maxim would be: Not nostrums but normalcy!"
80. Popper 1994, pp. 155–156: "It is my view that the methods of the natural as well as the social sciences can be best understood if we admit that science always begins and ends with problems. The progress of science lies, essentially, in the evolution of its problems. And it can be gauged by the increasing refinement, wealth, fertility, and depth of its problems. ... The growth of knowledge always consists in correcting earlier knowledge. Historically, science begins with pre-scientific knowledge, with pre-scientific myths and pre-scientific expectations. And these, in turn, have no 'beginnings'."
81. Popper 1963, p. 37: "[B]y making their interpretations and prophecies sufficiently vague [astrologers] were able to explain away anything that might have been a refutation of the theory had the theory and the prophecies been more precise. In order to escape falsification they destroyed the testability of their theory. It is a typical soothsayer's trick to predict things so vaguely that the predictions can hardly fail: that they become irrefutable."
82. Kuhn 1965, pp. 7-8: "Astrology is Sir Karl’s most frequently cited example of a 'pseudo-science'. He [Popper] says: 'By making their interpretations and prophecies sufficiently vague they [astrologers] were able to explain away anything that might have been a refutation of the theory had the theory and the prophecies been more precise. In order to escape falsification they destroyed the testability of the theory.' Those generalizations catch something of the spirit of the astrological enterprise. But taken at all literally, as they must be if they are to provide a demarcation criterion, they are impossible to support. The history of astrology during the centuries when it was intellectually reputable records many predictions that categorically failed. Not even astrology’s most convinced and vehement exponents doubted the recurrence of such failures. Astrology cannot be barred from the sciences because of the form in which its predictions were cast."

Abbreviated references

1. Grayling 2019, p. 397.
2. Popper 1963.
3. Uebel 2019.
4. Creath 2017.
5. Lakatos 1978, pp. 12–30.
6. Pera 1989, p. 362.
7. Maxwell 1974, p. 299.
8. Lakatos 1978, pp. 96–97.
9. Fine 2019.
10. Popper 1959, p. 32.
11. Thornton 2016, sec. 4.
12. Popper 1959, pp. 64–65.
13. Popper 1959, p. 65 Footnote *1.
14. Rudge 2005.
15. Theobald 2006.
16. Wallis 2005.
17. Dawkins 1995.
18. Dawkins 1986.
19. Dienes 2008, pp. 18–19.
20. Maxwell 1974, p. 294.
21. Keuth 2005, p. 45.
22. Maxwell 1974, p. 295.
23. Einstein 2010.
24. Burton 2000, p. 12–13.
25. Popper 1994b, Chap. 15.
26. McLean v. Arkansas 1982.
27. Daubert 1993.
28. Krafka 2002, p. 17 in archived pdf.
29. Popper 1972, chap. 2 sec. 12.
30. Popper 1972, appendix I.
31. Popper 1972, p. 89.
32. Popper 1972, chap 2, sec. 21.
33. Lakatos 1974, p. 245.
34. Popper 1974, p. 1010.
35. Kuhn 1996.
36. Martin 2017.
37. Feyerabend 1993.
38. Broad 1979.
39. Sokal & Bricmont 1998.
40. Miller 2000.

References

• Andersson, Gunnar (2016). "The Problem of the Empirical Basis in Critical Rationalism". In Shearmur, Jeremy; Stokes, Geoffrey (eds.). The Cambridge Companion to Popper. Cambridge Companions to Philosophy. Cambridge, UK; New York: Cambridge University Press. pp. 125–142. doi:10.1017/cco9781139046503.005. ISBN 9781139046503. OCLC 925355415.CS1 maint: ref=harv (link)
• Broad, W. J. (2 November 1979). "Paul Feyerabend: Science and the Anarchist". Science. 206 (4418): 534–537. Bibcode:1979Sci...206..534B. doi:10.1126/science.386510. ISSN 0036-8075. PMID 386510.CS1 maint: ref=harv (link)
• Burton, Dawn, ed. (2000). Research Training for Social Scientists: A Handbook for Postgraduate Researchers. Los Angeles: Sage Publications. ISBN 0-7619-6351-0.CS1 maint: ref=harv (link)
• Creath, Richard (2017) [First published 2011]. "Logical Empiricism". In Zalta, Edward N. (ed.). Stanford Encyclopedia of Philosophy (Fall 2017 ed.).CS1 maint: ref=harv (link)
• Couvalis, George (1997). The Philosophy of Science: Science and Objectivity. SAGE Publications. ISBN 978-0-7619-5101-8.CS1 maint: ref=harv (link)
• Cruzan, Mitchell B. (2018). Evolutionary Biology: A Plant Perspective. Oxford University Press. ISBN 9780190882686.CS1 maint: ref=harv (link)
• Darwin, Charles (1869). On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life (5th ed.). London: John Murray. Retrieved 22 February 2009.CS1 maint: ref=harv (link)
• Daubert v. Merrell Dow Pharmaceuticals, Inc. (US Supreme Court 1993). Text
• Dawkins, Richard (1986). The Blind Watchmaker. New York: W. W. Norton & Company, Inc. ISBN 0-393-31570-3.CS1 maint: ref=harv (link)
• Dawkins, Richard (1995). River Out of Eden. New York: Basic Books. ISBN 0-465-06990-8.CS1 maint: ref=harv (link)
• Dienes, Zoltan (2008). Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference. New York: Palgrave Macmillan. ISBN 978-0230542303. OCLC 182663275.CS1 maint: ref=harv (link)
• Duhem, Pierre (1906). La théorie physique: son objet et sa structure (in French). Chevalier & Rivière.CS1 maint: ref=harv (link)
• Duhem, Pierre (1991) [First published 1954]. The Aim and Structure of Physical Theory. Princeton University Press. ISBN 0-691-02524-X.CS1 maint: ref=harv (link)
• Einstein, Albert (2010) [First published 1923]. "Geometry and Experience". Sidelights on Relativity. New York: Dover Publications. p. 27. ISBN 978-0-486-24511-9.CS1 maint: ref=harv (link)
• "Federal Rules of Evidence 702 (Notes)". Cornell Law School. Legal Information Institute. April 26, 2011.
• "Federal Rules of Evidence" (PDF). United States Courts. Federal Judiciary of the United States. Archived from the original (PDF) on 2017-11-19. Retrieved November 19, 2017.
• Feldman, Burton; Williams, Katherine (2007). Williams, Katherine (ed.). 112 MERCER STREET: Einstein, Russell, Godel, Pauli, and the End of Innocence in Science. New York: Arcade Publishing.CS1 maint: ref=harv (link)
• Feyerabend, Paul (1978). Science in a Free Society. London: NLB. ISBN 0-86091-008-3.CS1 maint: ref=harv (link)
• Feyerabend, Paul (1978). "On the Critique of Scientific Reason". In Wartofsky, M.W.; Feyerabend, P.K.; Cohen, R.S. (eds.). Essays in Memory of Imre Lakatos. Boston Studies in the Philosophy and History of Science. 39. pp. 109–143.CS1 maint: ref=harv (link) CS1 maint: date and year (link)
• Feyerabend, Paul (1981). Problems of Empiricism: Volume 2: Philosophical Papers. Cambridge University Press, 1985. ISBN 9780521316415.CS1 maint: ref=harv (link)
• Feigl, Herbert (1978). "Positivism". Encyclopædia Britannica. Retrieved May 3, 2020.CS1 maint: ref=harv (link)
• Fisher, R.A. (1930). The genetical theory of natural selection. Рипол Классик. ISBN 978-1-176-62502-0.CS1 maint: ref=harv (link)
• Fine, Kit (2019). "Verisimilitude and Truthmaking". Erkenntnis. doi:10.1007/s10670-019-00152-z.CS1 maint: ref=harv (link)
• Grayling, A.C. (2019). The history of philosophy. New York: Penguin. ISBN 9780241304556. OCLC 1054371393.CS1 maint: ref=harv (link)
• Kaye, David H. (2005). "On 'Falsification' and 'Falsifiability': The First Daubert Factor and the Philosophy of Science". Jurimetrics. 45 (4): 473–481. JSTOR 29762910. SSRN 767086.CS1 maint: ref=harv (link)
• Keuth, Herbert (2005) [Published in German 2000]. The Philosophy of Karl Popper (1st English ed.). Cambridge, UK; New York: Cambridge University Press. ISBN 9780521548304. OCLC 54503549.CS1 maint: ref=harv (link)
• Krafka, Carol L.; Miletich, D. Dean P.; Cecil, Joe S.; Dunn, Meghan A.; Johnson, Mary T. (September 2002). "Judge and Attorney Experiences, Practices, and Concerns Regarding Expert Testimony in Federal Civil Trials" (PDF). Psychology, Public Policy, and Law. 8 (3): 309–332. Archived from the original on April 11, 2020.
• Kuhn, Thomas S. (1965) [Published 1970; reprinted Kuhn 1974]. "Logic of Discovery or Psychology of Research?". In Lakatos, Imre; Musgrave, Alan (eds.). Criticism and the Growth of Knowledge. Proceedings of the International Colloquium in the Philosophy of Science. 4. London: Cambridge University Press. ISBN 0521078261. OCLC 94900.CS1 maint: ref=harv (link)
• Kuhn, Thomas S. (1974) [First published Kuhn 1965]. "Logic of Discovery or Psychology of Research?". In Schilpp, Paul Arthur (ed.). The Philosophy of Karl Popper. II. Illinois: Open Court. pp. 798–819. ISBN 0875481426. OCLC 2580491.CS1 maint: ref=harv (link)
• Kuhn, Thomas S. (1996) [First published 1962]. The Structure of Scientific Revolutions (3rd ed.). Chicago: University of Chicago Press. ISBN 9780226458076. OCLC 34548541.CS1 maint: ref=harv (link)
• Lakatos, Imre (1974) [Reprinted Lakatos 1978, pp. 139–167]. "Popper on demarcation and induction". In Schilpp, Paul Arthur (ed.). The Philosophy of Karl Popper. I. Illinois: Open Court. pp. 241–273. ISBN 0875481418. OCLC 2580491.CS1 maint: ref=harv (link)
• Lakatos, Imre (1978). Worrall, John; Curry, Gregory (eds.). The Methodology of Scientific Research Programmes: Volume 1: Philosophical Papers (1980 ed.). Cambridge, UK: Cambridge University Press. ISBN 0-521-28031-1.CS1 maint: ref=harv (link)
• Martin, Eric (2017). "Science and Ideology". Internet Encyclopedia of Philosophy. Archived from the original on 2019-11-04. Retrieved 4 November 2019.CS1 maint: ref=harv (link)
• Maxwell, Grover (1974). "Corroboration without demarcation". In Schilpp, Paul Arthur (ed.). The Philosophy of Karl Popper. I. Illinois: Open Court. pp. 292–321. ISBN 9780875481418.CS1 maint: ref=harv (link)
• McLean v. Arkansas Board of Education (Eastern District of Arkansas January 5, 1982). Text
• Miller, David (2000). "Sokal and Bricmont: Back to the Frying Pan" (PDF). Pli. 9: 156–73. Archived from the original (PDF) on 2007-09-28.CS1 maint: ref=harv (link)
• Muehlenbein, M.P. (2010). Human Evolutionary Biology. Cambridge University Press.CS1 maint: ref=harv (link)
• Musgrave, Alan (1976). "Method or Madness?: Can the Methodology of Research Programmes Be Rescued From Epistemological Anarchism?". In Cohen, R.S.; Feyerabend, P.K.; Wartofsky, M. W. (eds.). Essays in Memory of Imre Lakatos. Dordrecht: D. Reidel. pp. 457–491.CS1 maint: ref=harv (link)
• Pennock, Robert T. (2000). Tower of Babel: The Evidence Against the New Creationism. A Bradford Book. MIT Press. doi:10.7551/mitpress/6870.001.0001. ISBN 978-0262661652. OCLC 39262003.CS1 maint: ref=harv (link)
• Pera, Marcello (1989). "Methodological Sophisticationism: A Degenerating Project". In Gavroglou, Kōstas; Goudaroulis, Yorgos; Nicolacopoulos, Pantelis (eds.). Imre Lakatos and Theories of Scientific Change. Boston Studies in the Philosophy of Science. 111. Dordrecht; Boston: Kluwer Academic Publishers. pp. 169–187. doi:10.1007/978-94-009-3025-4. ISBN 902772766X. OCLC 17982125.CS1 maint: ref=harv (link)
• Popper, Karl (1959). The Logic of Scientific Discovery (2002 pbk; 2005 ebook ed.). Routledge. ISBN 978-0-415-27844-7. Falsifiability at the Internet Archive.CS1 maint: ref=harv (link)
• Popper, Karl (1963). Conjectures and Refutations: The Growth of Scientific Knowledge (2002 ed.). London: Routledge. ISBN 978-0-415-28594-0.CS1 maint: ref=harv (link) Excerpt: Science as Falsification.
• Popper, Karl (1972). Objective Knowledge: An Evolutionary Approach (2003 ed.). New York: Oxford University Press. ISBN 978-0198750246.CS1 maint: ref=harv (link)
• Popper, Karl (1974). "Replies to my Critics". In Schilpp, Paul Arthur (ed.). The Philosophy of Karl Popper. II. Illinois: Open Court. pp. 961–1197. ISBN 0875481426. OCLC 2580491.CS1 maint: ref=harv (link)
• Popper, Karl (1978). "Natural Selection and the Emergence of Mind". Dialectica. 32 (3/4): 339–355. doi:10.1111/j.1746-8361.1978.tb01321.x. JSTOR 42970324.CS1 maint: ref=harv (link)
• Popper, Karl (1980). "Evolution". Letters. New Scientist. Vol. 87 no. 1215. Reed Business Information. p. 611.CS1 maint: ref=harv (link)
• Popper, Karl (1983) [Originally written in 1962]. Realism and the Aim of Science: From the Postscript to The Logic of Scientific Discovery. London; New York: Routledge. doi:10.4324/9780203713969. ISBN 0-415-08400-8. OCLC 25130665.CS1 maint: ref=harv (link)
• Popper, Karl (1994). Notturno, Mark A. (ed.). The Myth of the Framework: In Defence of Science and Rationality. London; New York: Routledge. doi:10.4324/9780203535806. ISBN 9780415113205. OCLC 30156902.CS1 maint: ref=harv (link)
• Popper, Karl (1994) [First edition 1945]. The Open Society and Its Enemies. Routledge 2012, Princeton University Press 2013.CS1 maint: ref=harv (link) CS1 maint: date and year (link)
• Popper, Karl (2009) [Manuscript 1933, Published in German 1979]. Eggers Hansen, Troels (ed.). The Two Fundamental Problems of the Theory of Knowledge. Translated by Pickel, Andreas. London; New York: Routledge. doi:10.4324/9780203371107. ISBN 9780415394314. OCLC 212627154.CS1 maint: ref=harv (link)
• Ridley, Mark (2003). Evolution (3rd ed.). Blackwell Publishing. ISBN 1-4051-0345-0.CS1 maint: ref=harv (link)
• Rosende, Diego L. (2009). "Popper on Refutability: Some Philosophical and Historical Questions". In Parusnikova, Zuzana; Cohen, Robert S. (eds.). Rethinking Popper. Boston Studies in the Philosophy of Science. Springer. pp. 135–154. doi:10.1007/978-1-4020-9338-8_11. ISBN 9781402093371. OCLC 260208425.CS1 maint: ref=harv (link)
• Rudge, David W. (2005). "The Beauty of Kettlewell's Classic Experimental Demonstration of Natural Selection". BioScience. 55 (4): 369–375. doi:10.1641/0006-3568(2005)055[0369:TBOKCE]2.0.CO;2.CS1 maint: ref=harv (link)
• Ruse, Michael (2010). Science and Spirituality: Making Room for Faith in the Age of Science. Cambridge, UK; New York: Cambridge University Press. doi:10.1017/CBO9780511676338. ISBN 978-0-521-75594-8. OCLC 366517438.CS1 maint: ref=harv (link)
• Sokal, Alan D.; Bricmont, Jean (1998) [Published in French 1997]. Fashionable Nonsense: Postmodern Intellectuals' Abuse of Science. New York: Picador. ISBN 0312195451. OCLC 39605994.CS1 maint: ref=harv (link)
• Thompson, N.S. (1981). "Toward a falsifiable theory of evolution.". In Bateson, P.G.; Klopfer, P.H. (eds.). Perspectives in ethology. 4. New York: Plenum Publishing. pp. 51–73.CS1 maint: ref=harv (link)
• Theobald, Douglas L. (2006). "29+ Evidences for Macroevolution: The Scientific Case for Common Descent, Version 2.87". The Talk.Origins Archive. Archived from the original on 2011-05-14. Retrieved 2020-04-21.CS1 maint: ref=harv (link)
• Thornton, Stephen (2007). "Popper, Basic Statements and the Quine-Duhem Thesis". Yearbook of the Irish Philosophical Society. 9.CS1 maint: ref=harv (link)
• Thornton, Stephen (2016) [First published 1997]. "Karl Popper". In Zalta, Edward N. (ed.). Stanford Encyclopedia of Philosophy (Summer 2017 ed.).CS1 maint: ref=harv (link)
• Uebel, Thomas (2019) [First published 2006]. "Vienna Circle". In Zalta, Edward N. (ed.). Stanford Encyclopedia of Philosophy (Summer 2017 ed.).CS1 maint: ref=harv (link)
• United States v. Prime, 220 F. Supp. 2d 1203 (Western District of Washington September 20, 2002).
• Urban, Patricia (2016) [First published 2012]. Archaeological Theory in Practice. Routledge. ISBN 9781351576192.CS1 maint: ref=harv (link)
• Waddington, C.H. (1959). "Evolutionary Adaptation". Perspectives in Biology and Medicine. Johns Hopkins University Press. 2 (4): 379–401. ISSN 1529-8795.CS1 maint: ref=harv (link)
• Wallis, Claudia (2005-08-07). "The Evolution Wars". Time Magazine. PMID 16116981.CS1 maint: ref=harv (link)
• Watkins, John (1970). "Against 'Normal Science'". In Lakatos, Imre; Musgrave, Alan (eds.). Criticism and the Growth of Knowledge. Proceedings of the International Colloquium in the Philosophy of Science. 4. London: Cambridge University Press. pp. 25–37. ISBN 0521078261. OCLC 94900.CS1 maint: ref=harv (link)
• Zahar, E. G. (1983). "The Popper-Lakatos Controversy in the Light of 'Die Beiden Grundprobleme Der Erkenntnistheorie'". The British Journal for the Philosophy of Science. 34 (2): 149–171. doi:10.1093/bjps/34.2.149. JSTOR 687447.CS1 maint: ref=harv (link)

Further reading

• Binns, Peter (March 1978). "The Supposed Asymmetry between Falsification and Verification". Dialectica. 32 (1): 29–40. doi:10.1111/j.1746-8361.1978.tb01300.x. JSTOR 42971398.CS1 maint: ref=harv (link)
• Blaug, Mark (1992). The Methodology of Economics: Or, How Economists Explain. Cambridge University Press. ISBN 978-0-521-43678-6.CS1 maint: ref=harv (link)
• Chalmers, Alan F. (2013). What Is This Thing Called Science? (4th ed.). Indianapolis: Hackett Publishing Company. ISBN 9781624660382. OCLC 847985678.CS1 maint: ref=harv (link)
• Corfield, David; Schölkopf, Bernhard; Vapnik, Vladimir (July 2009). "Falsificationism and Statistical Learning Theory: Comparing the Popper and Vapnik-Chervonenkis Dimensions". Journal for General Philosophy of Science. 40 (1): 51–58. doi:10.1007/s10838-009-9091-3. JSTOR 40390670.CS1 maint: ref=harv (link)
• De Pierris, Graciela; Friedman, Michael. "Kant and Hume on Causality". In Zalta, Edward N. (ed.). Stanford Encyclopedia of Philosophy (Winter 2013 ed.).CS1 maint: ref=harv (link)
• Derksen, A. A. (November 1985). "The Alleged Unity of Popper's Philosophy of Science: Falsifiability as Fake Cement". Philosophical Studies: An International Journal for Philosophy in the Analytic Tradition. 48 (3): 313–336. doi:10.1007/BF01305393. JSTOR 4319794.CS1 maint: ref=harv (link)
• Elena, Santiago F.; Lenski, Richard E. (2003). "Evolution experiments with microorganisms: the dynamics and genetic bases of adaptation". Nature Reviews Genetics. 4 (6): 457–469.CS1 maint: ref=harv (link)
• Elkana, Yehuda (2018). "Einstein and God". In Galison, P.L.; Holton, G.; Schweber, S.S. (eds.). Einstein for the 21st Century: His Legacy in Science, Art, and Modern Culture. Princeton University Press.CS1 maint: ref=harv (link)
• Ferguson, Christopher J.; Heene, Moritz (2012). "A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science's Aversion to the Null". Perspectives on Psychological Science. 7 (6): 555–561. doi:10.1177/1745691612459059. PMID 26168112.CS1 maint: ref=harv (link)
• Feyerabend, Paul (1993) [First published 1975]. Against Method: Outline of an Anarchistic Theory of Knowledge (3rd ed.). Verso. ISBN 978-0-86091-646-8.
• García, Carlos E. (2006). "Falsifiability". Popper's Theory of Science: An Apologia. Continuum Studies in Philosophy. London; New York: Continuum. pp. 38–83. ISBN 0826490263. OCLC 62742611.CS1 maint: ref=harv (link)
• Gawronski, Bertram; Bodenhausen, Galen V. (7 January 2015) [12 November 2014]. "Theory Evaluation". Theory and Explanation in Social Psychology. Guilford Publications. ISBN 978-1-4625-1848-7.CS1 maint: ref=harv (link)
• Henderson, Leah (2018). "The Problem of Induction". In Zalta, Edward N. (ed.). Stanford Encyclopedia of Philosophy (Summer 2018 ed.).CS1 maint: ref=harv (link)
• Hume, David (1896) [First published 1739]. A Treatise of Human Nature (PDF). Oxford: Clarendon Press. OCLC 779563. Falsifiability at the Internet Archive. Archived from the original (PDF) on 2019-08-10.CS1 maint: ref=harv (link)
• Johansson, Lars-Goran (2015). "Falsificationism". Philosophy of Science for Scientists. Cham: Springer. pp. 106–108. doi:10.1007/978-3-319-26551-3_6. ISBN 9783319265490. OCLC 923649072.CS1 maint: ref=harv (link)
• Kant, Immanuel (1787). Guyer, Paul; Wood, Allen W (eds.). Critique of Pure Reason. The Cambridge edition of the works of Immanuel Kant (1998 ed.). Cambridge, UK; New York: Cambridge University Press. doi:10.1017/cbo9780511804649. ISBN 9780521354028. OCLC 36438781.CS1 maint: ref=harv (link)
• Kasavin, Ilya; Blinov, Evgeny (2012). "Hume and Contemporary Philosophy: Legacy and Prospects". In Ilya Kasavin (ed.). David Hume and Contemporary Philosophy. Cambridge Scholars. pp. 1–9. ISBN 9781443841313. OCLC 817562250. Archived from the original on 2016-09-17.CS1 maint: ref=harv (link)
• Koterski, Artur (2011). "The Rise and Fall of Falsificationism in the Light of Neurath's Criticism". In Dieks, Dennis Geert Bernardus Johan; Gonzalez, Wenceslao J.; Hartmann, Stephan; Uebel, Thomas; Weber, Marcel (eds.). Explanation, Prediction, and Confirmation. Philosophy of Science in a European Perspective. 2. New York: Springer. pp. 487–498. doi:10.1007/978-94-007-1180-8_33. ISBN 9789400711792. OCLC 706920414.CS1 maint: ref=harv (link)
• Lange, Marc (2008). "Hume and the Problem of Induction". In Gabbay, Dov M.; Woods, John (eds.). Inductive Logic. Handbook of the History of Logic. 10. Amsterdam; Boston: Elsevier. pp. 43–91. CiteSeerX 10.1.1.504.2727. ISBN 9780444529367. OCLC 54111232.CS1 maint: ref=harv (link)
• Maxwell, Nicholas (2017). "Popper, Kuhn, Lakatos and Aim-Oriented Empiricism". Karl Popper, Science and Enlightenment. London: UCL Press. pp. 42–89. doi:10.14324/111.9781787350397. ISBN 9781787350397. OCLC 1004353997.CS1 maint: ref=harv (link)
• McGinn, Colin (2002). "Looking for a Black Swan". The New York Review of Books (November 21, 2002): 46–50.CS1 maint: ref=harv (link)
• Merritt, David (February 2017). "Cosmology and convention". Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics. Elsevier. 57.CS1 maint: ref=harv (link)
• Miller, David (1994). Critical Rationalism: A Restatement and Defence. Chicago: Open Court. ISBN 9780812691979. OCLC 30353251.CS1 maint: ref=harv (link)
• Miller, David (2006). "Falsifiability: More Than a Convention?". Out of Error: Further Essays on Critical Rationalism. Aldershot, UK; Burlington, VT: Ashgate. pp. 81–110. ISBN 9780754650683. OCLC 57641308.CS1 maint: ref=harv (link)
• Miller, David (2014). "Some Hard Questions for Critical Rationalism". Discusiones Filosóficas. 15 (24): 15–40. ISSN 0124-6127.CS1 maint: ref=harv (link)
• Niiniluoto, Ilkka (1984) [Chapter first published 1978]. "Notes on Popper as Follower of Whewell and Peirce". Is Science Progressive?. Synthese Library. 177. Dordrecht; Boston: D. Reidel. pp. 18–60. doi:10.1007/978-94-017-1978-0_3. ISBN 9027718350. OCLC 10996819.CS1 maint: ref=harv (link)
• Popper, Karl (1976). Bartley III, William W. (ed.). Unended Quest: An Intellectual Autobiography (2002 ed.). London and New York: Routledge. ISBN 0415285895.CS1 maint: ref=harv (link)
• Popper, Karl (26 February 1982). "Les chemins de la verite: L'Express va plus loin avec Karl Popper". L'Express (Interview). Interviewed by S. Lannes and A. Boyer. pp. 82–88.CS1 maint: ref=harv (link) CS1 maint: date and year (link)
• Popper, Karl (1989). "Zwei Bedeutungen von Falsifizierbarkeit [Two meanings of falsifiability]". In Seiffert, H.; Radnitzky, G. (eds.). Handlexikon der Wissenschaftstheorie [Dictionary of epistemology] (in German) (1992 ed.). München: Deutscher Taschenbuch Verlag. ISBN 3-423-04586-8.CS1 maint: ref=harv (link)
• Popper, Karl (1992) [Originally written in 1962]. Bartley III, W.W. (ed.). Quantum Theory and the Schism in Physics: From the Postscript to the Logic of Scientific Discovery (2005 ed.). London; New York: Routledge. doi:10.4324/9780203713990. ISBN 0415091128. OCLC 26159482.CS1 maint: ref=harv (link)
• Rescher, Nicholas (1977). "Confirmationism vs. Falsificationism". Dialectics: A Controversy-Oriented Approach to the Theory of Knowledge. Albany: State University of New York Press. pp. 119–123. ISBN 087395372X. OCLC 3034395.CS1 maint: ref=harv (link)
• Rescher, Nicholas (1989). "Generality Preference and Falsificationism". Cognitive Economy: The Economic Dimension of the Theory of Knowledge. Pittsburgh: University of Pittsburgh Press. pp. 118–123. ISBN 0822936178. OCLC 19264362.CS1 maint: ref=harv (link)
• Watkins, John (1974). "The Unity of Popper's Thought". In Schilpp, Paul Arthur (ed.). The Philosophy of Karl Popper. I. Illinois: Open Court. pp. 371–412. ISBN 0875481418. OCLC 2580491.CS1 maint: ref=harv (link)