Feeling the Future

"Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect" is a paper submitted to and published in the Journal of Personality and Social Psychology by Daryl J. Bem of Cornell University that claims to offer strong evidence for the case of precognition in humans.[1] It gained some notoriety during its "in press" phase in late 2010, with several pop science organisations and blog posts covering the experiment and attempting to understand it.

Putting the psycho in
Parapsychology
Men who stare at goats
By the powers of tinfoil
v - t - e

Methodology

The paper introduces some ingenious methodologies for rigorous study of certain kinds of psychic claims. The ingenious part comes from taking well-established psychological tests and reversing the order that the individual tests are administered. For instance, in one test participants are asked to type certain words, selected at random from a list, and then are asked to recall words from the full list. The result is that they are capable of recalling more of the words that they had previously typed. In Bem's research, this is turned on its head: students were asked to recall words from a list and then asked to type a random selection of them. The result being that these students recalled the words that they would later type — the ones selected at random that they had no prior knowledge of at the time of the recall test. This suggests that their minds were somehow aware of what words they would experience in the future and selected them from the full list in advance!

However, the statistical analysis and subject selection criteria are highly suspect and the statistically significant effects are likely a product of experimenter bias.

Specific criticisms

A response by E.J. Wagenmakers et al. highlights some of the major issues that call into question the validity of the analysis by Bem.[2]

  • Bem has published his own research methodology and encourages the formulation of hypotheses after data analysis. This form of post-hoc analysis makes it very difficult to determine accurate statistical significance. It also explains why Bem offers specific hypotheses that seem odd a priori, such as erotic images having a greater precognitive effect. Constructing hypotheses from the same data range used to test those hypotheses is a classic example of the Texas sharpshooter fallacy
  • The paper uses the fallacy of the transposed conditional to make the case for psi powers. Essentially mixing up the difference between the probability of data given a hypothesis versus the probability of a hypothesis given data.
  • Wagenmakers' analysis of the data using a Bayesian t-test removes the significant effects claimed by Bem.

Any new result in any field of science requires extensive independent replication before it can be accepted as valid. This is no different for psychic powers, drug studies, or theories of physics. At least one replication of one of the tasks Bem used has failed to show significance,[1] and hopefully others will attempt additional replications. Researcher Richard Wiseman is attempting to create a registry of replication attempts[3] to control for publication bias.

In the end, the paper offers some promising methods but should not be taken as evidence for precognition without extensive independent replication of the results, particularly since there are significant questions about the introduction of bias into the analysis of the data.

An analysis published in Psychonomic Bulletin & Review found statistical evidence of publication bias in Bem's reported results.[4]

In March of 2012 three independent failed replication attempts were reported in a PLoS ONE paper.[5] The paper also noted a failed replication published in the Journal of the Society for Psychical Research. Bem tried to brush it off by suggesting that because the group attempting replication was skeptical this would negatively influence the results.[6] According to the paper Bem presented at the Parapsychological Association 46th Annual Convention in 2003 he had actually already tested for this possibility and found that a researcher's skepticism had no such effect.[7] The group had substantial problems trying to get their negative results published, several prominent journals refused to consider publishing a replication and another journal rejected the paper after a negative report from one referee was later confirmed to be Daryl Bem himself.[8]

A test of Bem's "retroactive priming" published in Memory & Cognition found no evidence of any such effect.[9]

Bem has published a detailed response to Wagenmakers et al. in Journal of Personality and Social Psychology.[10] Bem et al respond to other critics and present a meta-analysis of 90 experiments in a paper currently under review.[11]

gollark: Well, all potatOS ones, so about three.
gollark: Superglobals are like globals, but shared across all computers everywhere.
gollark: Including the superglobals™ function which powers the string metatable bug reimplementation.
gollark: A lot of potatOS runs in the CLOUD™, by which I mean random people's web APIs.]
gollark: Yes, it uses the TryHaskell website's API to "run" "Haskell".

See also

Further reading

Unpublished failed replications:

Bem's 2003 and 2005 preliminary reports on these experiments at the Parapsychological Association Convention:

References
  1. New Scientist, "Is this evidence that we can see the future?", Peter Aldhous, 11 November 2010.
  2. "Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi: Comment on Bem (2011)" doi:10.1037/a0022790
  3. Are you conducting a replication of Daryl Bem's 'Feeling the Future' experiments?
  4. Francis G (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review 19(2): 151–156. doi:10.3758/s13423-012-0227-9.
  5. Ritchie SJ, Wiseman R, French CC (2012). Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect. PLoS ONE 7(3): e33423. doi:10.1371/journal.pone.0033423
  6. Bem's psychic study cast into doubt as it fails replication attempts, Doubtful News.
  7. Bem, D. J. (2003). Precognitive habituation: Replicable evidence for a process of anomalous cognition. Proceedings of Presented Papers: The Parapsychological Association 46th Annual Convention. 6-20.
  8. Chris French (15 March 2012). Precognition studies and the curse of the failed replications. The Guardian.
  9. Feeling the past: The absence of experimental evidence for anomalous retroactive influences on text processing.
  10. Bem D (2011). Must Psychologists Change the Way They Analyze Their Data? Journal of Personality and Social Psychology 101(4), 716–719 doi: 10.1037/a0024777.
  11. Bem D et al.(2014) Feeling the Future: A Meta-analysis of 90 Experiments on the Anomalous Anticipation of Random Future Events. Under editorial review.
This article is issued from Rationalwiki. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.