Asset Publisher

Back 2017_03_03_opinion_ICS_lopez_fidalgo

Jesus Fernando Lopez Fidalgo, Catedrático de Estadística. Instituto Cultura y Sociedad, Universidad de Navarra

The keys to quality research

Several researchers recently published a manifesto in Nature criticizing current scientific research’s lack of reliability (http://www.nature.com/articles/s41562-016-0021), which has long worried members of the scientific community.

lun, 27 feb 2017 18:06:00 +0000 Publicado en El Español

Other studies show that around 2 million articles are published worldwide each year. Each article is read by an average of just 10 people, including the co-authors, editors and reviewers involved in its publication. 82% of the articles in the humanities do not receive a single citation and authors only actually read about 20% of the articles they cite. There is a great deal of pressure to publish quickly and in abundance, ostensibly leading to this disaster. But this is not as new of a problem as it might seem. As some have noted, years ago, of the ten most relevant articles in the history of mathematics, seven of them were rejected at first.

In any case, a study cannot be replicated 100%. Only mathematical results demonstrated with the laws of logic and based on perfectly defined and unappealable principles or assumptions can be said to be completely reproducible. For example, the proof that every continuous line with negative and positive values ​​must necessarily equal zero at some point is infallible. No matter how many times we reproduce the experiment drawing continuous lines that start in the negative area and end in the positive area, they will always be forced to pass through zero. The experiment will never fail. But almost everything else is reproducible with some probability. That results are replicated in front of a group of experts provides assurance that the data is a guarantee, but it does not add certainty.

On the other hand, recent scientific research has, for example, shown that life expectancy has grown dramatically over the last several years and that certain scientific advances very clearly contribute to well-being. That is why I am not an alarmist, an unproductive posture indeed. For research to reveal anything relevant, researchers must try at whatever they are studying many times and therefore produce research that will never be relevant, even if it has been done in good faith. This does not justify doing things poorly. The abovementioned manifesto gives some clues as to what good research looks like, which seems to me a very important and laudable contribution.

From a statistical-methodological point of view, the following basic principles, among others, should be taken into account when attempting to undertake research correctly. The hypotheses to be demonstrated must be established at the beginning— not even before collecting the data, but rather before planning data collection itself. Precisely this order of things allows for an adequate and efficient data collection plan (experimental design or sampling). Experiments, surveys, etc. must be carried out with the supervision of the person who designed the plan. Correct treatment of non-responses or other difficulties in data collection are key in any given study. When analyzing the data, supporting hypotheses must be verified instead of accepting the results of automatic statistical software without question. For example, if we collect data about literature preferences, we could code the answers as 1 (science fiction), 2 (essays), 3 (historical), however, statistical software will calculate the average of these codes if we ask it to do so, but it will not supply any meaning.

No one consciously does this, but these kinds of more subtle errors are out there and slide by if researchers lack the knowledge or adequate assistance to avoid them. When submitting results in a publication, the problems and difficulties that arise during the process and anything else that helps in evaluating the study should be clearly provided.

The statistical model that is employed never fits perfectly with reality and therefore must always be used with due precaution. George Box (d. 2013), who was one of the most prominent statisticians of our time, used to say that all models are fake, but some are useful. At least statistical models are able to specify their level of error, always under the basic assumptions of the model in use. Other procedures, for example, algorithms based on nature, work in practice, but do not control for error. That does not mean they should not be used. In fact, they are used, for example, for web-based search engines or for automatic translators and they work, but should always be employed with care.

We must continue researching a great deal and as best as we can. Incredible progress has been made in our country. Rather than just talk about it, we must invest in research with action and with tangible funding to ensure the next generation of researchers. We must also quietly rethink research evaluation procedures without lowering the bar or listening to false excuses.