Image by Nathan Osman on Pixabay.
I just read that about 57 million people watching the US Presidential debate. (Other sources had 67 million but who knows?) It was in a post from Morning Brew — it’s free and you can subscribe here. Bite-size news chunks instead of lengthy news from the papers. Just what you want and no more. Who needs the Times?
Anyway, the article said this number was “significantly” greater than the. 49 million who watched the previous debate,
And I have a problem with this. What does the word “significantly” mean here?
There is “statistical significance.” That means the likelihood of this event occurring by chance is low – usually less than 5%. You can get a more detailed explanation of statistical significance here.
In the medical world, a doctor once introduced me to “clinical significance.” That means the results make a difference in a clinical setting. The result is useful in a clinical setting.
It’s usually a subjective measure. However, that doctor once told me to ignore differences between the control group (usually a placebo effect) and a treatment group (the ones who got the drug) if the difference was less than 3%.
In other words, you can have a difference that is statistically significant, but you can ignore it. It’s explained in more detail in this article.
Finally, I believe there’s something I call journalistic significance. I just made up that term but I think that’s what Morning Brew is using in that article. Significance here means you have a certain “wow” factor. It’s entirely subjective.
Some scientists, especially medical people, become quite defensive. I once was talking to an academic researcher about very small differences in an outcome. She sniffed, “Well, it’s statistically significant,” as if that ended the discussion. In another setting, it would be a dismissive answer, almost gaslighting me.
I don’t think she’d ever heard the term “clinical significance.” As I well know, it’s statistical significance that gets you published.
The bottom line. When someone tells you something is “significant,” ask how they’re using that term. When someone tells you something is “significant,” ask what they mean. And I’ll bet you’ll get someone scratching their head and muttering, “I dunno…”
