<

Speculation vs. science: not all studies are created equal (in Mindcraft Stories)

I wrote for the science magazine Mindcraft Stories about how quality studies differ from knee-jerk studies and why it's important not to jump to conclusions from small sample research.

Especially now, when so many people are sharing social media links directly to academic journals, which they consider equally valid just because they "sound scientific", I thought it was important to explain the mechanisms by which the scientific community decides what is credible and what is not.

In communication, researchers are being asked by the press and the public to drop the nuances, to stop being so cautious, to risk a forecast. But in these times, I think the reverse is the case: the public needs to understand what the limitations of a study mean, what small samples imply, what it means to not have your conclusions validated by peer review and where pure speculation starts.

The full article is here: https://mindcraftstories.ro/index.php/2020/03/26/speculatii-vs-stiinta-nu-toate-studiile-sunt-egale/

A short excerpt:

"Since the pandemic started, thousands of opinions, expert opinions, studies, reports and scientific articles about the Coronavirus have appeared on Facebook, WhatsApp, Instagram and other channels.

In this period, the phrase "scientists say that" often finds its place in these more or less official reports. Unfortunately, behind it lies not only correct information, but also misinformation, speculation or manipulation.

Articles published in scientific journals are posted en masse, but they have to be read differently from the usual articles published by journalists. In addition, not all studies are equally meticulous, so not all are of the same scientific quality. Here's how researchers distinguish reliable studies from knee-jerk ones, so they don't jump to conclusions. 

First clue: does the study appear in a major scientific journal?

Prestigious scientific journals have their own ways of discerning whether a study is done correctly or not. When a researcher says they have discovered something, before the journal publishes that finding, they are obliged to send the information to other researchers in the field to evaluate it. They have to say whether the method is correct, whether the conclusion is reasonable or whether they see potential errors. This process is called "peer reviewing" and is the scientific community's way of deciding what is science and what is not. It is not a flawless method, but it is the best available at this time.

Because a paper published in such a scientific journal passes through many hands, it carries more weight than one published without having been checked.

Second clue: how highly rated is the journal?

The forest of scientific journals is full of dry patches, but there is a mechanism to prioritise them. Each publication is given some ratings, the equivalent of hotel stars. This grade is called the "impact factor" and basically counts how many citations the articles published there have over a longer period. The more often an article is cited in new research, the more useful it is considered to have been. Based on the impact factor, journals are ranked and the most prestigious ones are in the top quarter of the ranking. 

How do you check the magazine's "score"? The Scimago website displays a full report. If a journal has the Q1 indicator, it means it is among the best 25% scientific journals in that field.

There is a second check that can be done. Because the impact factor can be artificially increased, a measure of journal influence - Article Influence Score - is also used. The most prestigious journals will have a score of 100%."

Read the rest of the text here.

Every month, I send a newsletter about Science & Communicationwith articles on how research can be better promoted in Romania. Includes what I write here on the blog, but also recent articles in other languages that I recommend and comment on.
You can subscribe here.

CategoriesUncategorised

Leave a Reply

Your email address will not be published. Required fields are marked *