Redefine Statistical Significance

Add to Calendar
Date and Time: 
Monday, September 18, 2017
2:00 pm – 3:30 pm
For claims of discoveries of novel effects, we advocate a change in the P-value threshold for a “statistically significant” result from 0.05 to 0.005. Results currently called “significant” that do not meet the new threshold would be called suggestive and treated as ambiguous as to whether there is an effect. We highlight a fact that statisticians have known for a long time but which is not widely recognized in many scientific communities: evidence that is statistically significant at P = 0.05 actually constitutes fairly weak evidence. Many problems (such as multiple hypothesis testing and low power) have contributed to a high rate of false positives in many communities, and we emphasize that it is important to address all of these problems. We argue, however, that tightening the standards for statistical significance is a simple step that would help. We discuss and respond to several potential objections to the proposal.