But what is genuinely most fascinating, at least to me […] is the strange way it lets you write encyclopedia pagesâ€”the structures that have built up since its founding in 2001. The way that Wikipedia is composed is a good example of what happens when you build something so incredibly simple that anyone can use it, and then everyone does.
In many fields of research right now, scientists collect data until they see a pattern that appears statistically significant, and then they use that tightly selected data to publish a paper. Critics have come to call this p-hacking, and the practice uses a quiver of little methodological tricks that can inflate the statistical significance of a finding. As enumerated by one research group, the tricks can include:
- â€œconducting analyses midway through experiments to decide whether to continue collecting data,â€
- â€œrecording many response variables and deciding which to report postanalysis,â€
- â€œdeciding whether to include or drop outliers postanalyses,â€
- â€œexcluding, combining, or splitting treatment groups postanalysis,â€
- â€œincluding or excluding covariates postanalysis,â€
- â€œand stopping data exploration if an analysis yields a significant p-value.â€
Add it all up, and you have a significant problem in the way our society produces knowledge.