Remember when the Polytechnic University of Bucharest announced that they were testing an earthquake prediction system 4 hours in advance? And it wasn't greeted with laughter? (The article was written in 2019. Update 22.03.2023: The founder of the partner company has left the company. The Twitter account of the company remains out of date.)

Science communicators can easily fall into the trap of information that looks scientific but isn't. What is the difference between a scientific result that is well presented and appealing to the general public and a news story about a sensational, gold-winning invention at the Geneva show? Why are the statements of a scientist who says there could be water on Mars taken seriously, but the statements of someone who says cars will run on water are mocked?

This text is is intended to be a basic information assessment tool to decide whether something is credible enough to be turned into either a subject of a press release (by a press office), or a news article (for journalists).

The explanations come from my own experience, with the caveat that in addition to these basic rules I'm talking about here, it's worth going into more detail towards more nuanced criteria, and I promise this post will have a second part.

I will exemplify the ideas on the earthquake prediction news, because it is an interesting case: it may be pseudoscience, it may not. I'll also say, at the end, what I would have done in the same situation and how it was ommunicated on the same topic.

First here are the three filters through which I pass science information.

1. The information needs to come from a credible person/institution
2. The information needs to be backed by a study
3. The information must be independently verified, in case of doubt

Let's look at each of those.

What does "person /credible institution" mean?

In general, institutional sources - meaning universities, research institutes and research or companies - can be considered credible as long as their bias and their intention to promote themselves are taken into consideration. In the field of research, several institutions have taken a serious public role as providers of quality scientific information and perhaps promotion matters less. In the case of companies I would be more sceptical and add a verification mechanism of the information. Also, a news story coming from a press office means it would have to go through the filters of some communicators experienced communicators who can't afford to convey distorted information.

A credible person means that the person issuing the main information must be an expert in his/her field. In science, this would mean citing the work of a reputable researcher, the minimum requirement being to have a demonstrated interest in that field (research projects, articles scientific papers, conferences or a course on the subject). This is a simplified view. There is also a hierarchy and 'scores' for researchers, and some people are more able than others to talk about a topic, but I will go into details another time. For now it is enough: let there be one person who speaks about his or her work.

Let's see this applied to the news about earthquake prevention. The information seems to have come from a credible source: the Polytechnic University of Bucharest is prestigious, and its rector can certainly be considered credible. But on closer examination, we note that the rector did not represent his specialisation, having been trained in economic engineering, but only the institution; the information, although published on the institution's website, was originally sourced from an interview for press agency Agerpres from a conference organised by UPB together with a private company (Ionoterra). I'll explain immediately why this should raise questions.

What does it mean to be backed by a study?

Even credible individuals or institutions can be viewed with circumspection when they issue messages without clearly stating the where the information comes from. Is it the result of a research project? In what project and who funded it? The results are confirmed by publications in academic journals? In which journals were they published? Were the results presented at a conference? What were the reactions?

It's important that scientific information has been originally published in a journal - the more highly rated, the more credible the information. Why? Because publishing results in a journal means that those results, the way researchers arrived at them, any errors in reasoning have been seen by other researchers in the field, through peer review system. In other words, good academic journals do not show articles automatically as soon as they are submitted, but they are analysed, evaluated often modified and only then published.

Again, here too there are "rankings" of credibility of scientific journals, impact scores and other factors about I will talk about in a separate post. And one more thing: not infrequently, it has been shown that even very large journals have published false results and there is no infallible guarantee, but such cases are extremely rare. As a general rule, a respectable scientific journal will only publish results that were verified.

How does this apply to the earthquake prevention news? The information presented was not based on a study, but on the presentation of the company Ionoterra which promises to have the necessary technology. On the company's website there is an explanation of the technology, as well as a description of predictions and monitoring made at a laboratory of the University of Balti. There are references to a few published studies by one of the founders, Nathan Blaunstein, but as mentioned above it is important to see which journal accepted them for publication. In the case of the study cited first on the website, this is a 2009 article from the journal Physics and Chemistry of the Earth Parts A/B/C, a journal ranked below the middle of the scientific journals in the field. Moreover, the article was poorly cited by other researchers, but even some of those who referenced the study were sceptical.

Even so, the news would formally meet the criterion. There are some research results that would support this technique, which a journal agreed to publish ten years ago. Further.

What does independent verification mean?

Even within the press offices of research institutions, there is a process of evaluating information in terms of its attractiveness to the public, but also of what can or cannot be communicated: can we say that we are "first" which? Can we say it is "a first"? In science newsrooms, journalists usually ask for another opinion. To whom? To someone else in the field, from a different institution than the one from which the information originated. In Romania, it is not yet the practice to verify information from a researcher with an independent source, but the reason is not the convenience of journalists, but the fact that there is no culture of communication research news to the public.

How to find an independent researcher in the same field? One simple way is to look for experts in the same field from other universities/institutions of research institutions. Another way is to see what other scientific articles the published on the same topic and who has cited them. It is likely that people who have referenced the researcher's previous work are on topic and are still up to date with what he or she is doing.

Back to the case of the earthquake prevention news. The doubt here was obvious and should have immediately triggered caution in communication, especially on the part of those who issued the message. I think the journalists did their job and asked for another point of view, but probably didn't get it in time. The proof is that in response to numerous requests, a few days later (very late in press terms) The National Institute for Research and Development in Earth Physics (INCDFP) has issued an official statement - which is a very rare case in terms of institutional diplomacy.

We appreciate and respect the concerns of any public or private entity for knowledge and characterization of natural phenomena, but we recommend those who make such claims to show a responsible attitude towards the development and dissemination in the public space of results based on invalid experiments and / or partial or irrelevant measurements and data that can generate panic among the population.

How could we communicate better?

The problem is well identified in that reaction from INCDFP: indeed, private companies, and state institutions, are free to investigate; but communicating something before the result is a risky attitude and one that, in this case, has done more harm than good to the image of the University.

I also wondered how I would have reacted if I were the communicators. Science communication is a field in its infancy and it doesn't work as simply as communicating about socks or soft drinks. Training and awareness is still needed, from communicators to teachers, journalists and for those in decision-making positions. (Here you can see what I've done so far for that)

Strictly speaking, the information was correctly conveyed: a partnership was announced, not an outcome. This is a company that says it has the technique, the University does not guarantee that it works. The technique itself is one that has been tried and tested, by the European Space Agencyso not a crazy invention. However, the result was the opposite of what was intended.

Even if all the information was technically correct, I would still strongly advise that it should only be made public once there are concrete and verified results. That would have meant in the short term missing an image opportunity, but in the long term preserving credibility. An alternative strategy is to convey to the public an image of an "entrepreneurial university" that is willing to take big risks and to investigate ideas that initially seem strange, like an exotic start-up. But this requires the intention and strategy to communicate in this way, which clearly has not been the case now.

I was telling you that these tinkering with changes in the ionosphere are not so crazy and that this idea has been discussed before. How was it communicated on the same subject in Science? Can electrical signals in Earth's atmosphere predict earthquakes? Much more nuanced, with numerous caveats, pro and con opinions.

Perhaps a year from now the company's technology will prove a success that research mainstream did not foresee it. It would have been (and will be) an excellent story. We'll be reading about it, for sure, on the front page of the New York Times, including about the Polytechnic University of Bucharest, whose bet would have been a winner. Told too soon, however, it was a joke.