Statistical significance
R vs Python
Data science and its application
Statistical Significance: A Controversial Tool in Psychology Research
For many years, the domain of mathematics misled researchers with a method that, while straightforward, was fraught with potential misinterpretations. Mid-twentieth-century psychology, amidst the wake of hard sciences' achievements such as the Manhattan Project and early space race, was looking for definitive, universally valid outcomes.Psychologists turned to statistics to substantiate their intricate and often elusive findings, yearning for a mathematical foundation to draw empirical inferences. Thus, the concept of "statistical significance" was proposed, resulting in the adoption of a metric known as the 'p-value'.
The p-value is a calculated quantity from the experimental outcomes of each research study. When researchers found this value statistically significant, they believed they'd obtained credible results. This new approach was widely embraced, and in a relatively short period, numerous researchers reported statistically significant findings.
Gradually, psychology journals adopted a policy of publishing articles only if they presented statistically significant results. This led to an alarming increase in data manipulation and even cheating, to attain p-values less than 0.05 and secure publication slots.
In a bid to discourage this trend, Geoffrey Loftus, editor of the journal Memory & Cognition from 1993 to 1997, and a professor at the University of Washington, advised researchers to perform their calculations meticulously. Loftus urged researchers to focus more on correct results rather than statistical significance and recommended directly reporting averages from statistical findings. This advice would allow for more effective comparison of results across different studies.
Despite Loftus's efforts, statistical significance continued to be a prominent part of research reporting. Loftus later reflected that statistical significance aims to show what the world isn't based on, rather than offering insights into the world's true nature.
In the mid-twentieth century, notable psychology textbook authors carried out significant experiments using this controversial set of statistical principles. The results were accepted and regarded by researchers in other fields, such as social sciences, medical research, epidemiology, neuroscience, and biological anthropology, extending the influence of these principles far beyond their original context.
Scientists began to question their reliance on securing results without an integrated theory that could verify the accuracy of their predictions. This led them to explore various human-related subjects around statistical significance, often repeating prior processes, which gave a false sense of security.
The persistent focus on the p-value inhibits researchers from investigating theories that offer specific, high-risk predictions – elements crucial for testing a theory's validity. Rejecting the null hypothesis doesn't provide the researcher with new information; it only creates an opportunity to speculate about the cause of the observed effect.
As a result of the numerous failures in confirming statistically significant results, some journals now require researchers to submit comprehensive research designs and data before their articles are evaluated. This is to prevent data manipulation and increase the chances of publishing verifiable results.
Gerd Gigerenzer, director of the Harding Risk Literacy Center in Berlin, suggests that the central issue lies in the null hypothesis itself. Early pioneers like Wolfgang Köhler, Jean Piaget, and Ivan Pavlov, who developed influential theories, often conducted multiple studies using simple statistical methods that were later validated.
In the mid-20th century, psychologists, keen to validate their discipline scientifically, sought a diagnostic tool for their findings. This led to an emphasis on p-value, often at the expense of embracing a more complex statistical approach.
Statistical significance tests can not conclusively determine the reality of an event, leading to widespread misinterpretation. Studies have revealed that many researchers mistakenly believe that disregarding statistical significance implies non-existence of natural phenomena.
In a bid to discourage this trend, Geoffrey Loftus, editor of the journal Memory & Cognition from 1993 to 1997, and a professor at the University of Washington, advised researchers to perform their calculations meticulously. Loftus urged researchers to focus more on correct results rather than statistical significance and recommended directly reporting averages from statistical findings. This advice would allow for more effective comparison of results across different studies.
Despite Loftus's efforts, statistical significance continued to be a prominent part of research reporting. Loftus later reflected that statistical significance aims to show what the world isn't based on, rather than offering insights into the world's true nature.
In the mid-twentieth century, notable psychology textbook authors carried out significant experiments using this controversial set of statistical principles. The results were accepted and regarded by researchers in other fields, such as social sciences, medical research, epidemiology, neuroscience, and biological anthropology, extending the influence of these principles far beyond their original context.
Scientists began to question their reliance on securing results without an integrated theory that could verify the accuracy of their predictions. This led them to explore various human-related subjects around statistical significance, often repeating prior processes, which gave a false sense of security.
The persistent focus on the p-value inhibits researchers from investigating theories that offer specific, high-risk predictions – elements crucial for testing a theory's validity. Rejecting the null hypothesis doesn't provide the researcher with new information; it only creates an opportunity to speculate about the cause of the observed effect.
As a result of the numerous failures in confirming statistically significant results, some journals now require researchers to submit comprehensive research designs and data before their articles are evaluated. This is to prevent data manipulation and increase the chances of publishing verifiable results.
Gerd Gigerenzer, director of the Harding Risk Literacy Center in Berlin, suggests that the central issue lies in the null hypothesis itself. Early pioneers like Wolfgang Köhler, Jean Piaget, and Ivan Pavlov, who developed influential theories, often conducted multiple studies using simple statistical methods that were later validated.
In the mid-20th century, psychologists, keen to validate their discipline scientifically, sought a diagnostic tool for their findings. This led to an emphasis on p-value, often at the expense of embracing a more complex statistical approach.
Statistical significance tests can not conclusively determine the reality of an event, leading to widespread misinterpretation. Studies have revealed that many researchers mistakenly believe that disregarding statistical significance implies non-existence of natural phenomena.
Ali Reza Rashidi
Ali Reza Rashidi, a BI analyst with over nine years of experience, He is the author of three books that delve into the world of data and management.
