[ad_1]
Three men and a 97-year-old woman registered as “pregnant”, a 134-year-old patient, another diagnosed on May 50, 2020, two patients with a negative hospital “stay”, and 19 confirmed cases prior to the first official case diagnosed in Portugal. These are just some of the many problems identified in a study conducted by a team of 12 researchers from the Faculty of Medicine of the University of Porto (FMUP) on data from the Portuguese health authorities on covid-19. This Monday, the General Director of Health, Graça Freitas, justified that these data are from “epidemiological surveillance”, recognizing that they are not perfect, but highlighting that “the main priority now is to detect patients, treat patients and isolate contacts.”
“We describe some important quality problems in the Portuguese covid-19 surveillance databases, which may compromise the validity of some analyzes, with possible serious implications in a pandemic context,” write the authors of the article published on November 5 in the medRxiv platform, which publishes work that has not yet been peer-reviewed. Speaking to the PUBLIC, Cristina Costa Santos, researcher at the FMUP and one of the authors of the article that has already been sent for publication in the magazine Journal of epidemiology and community health, confirms the conclusion about the “poor quality” of the data analyzed and that they are based on the National Epidemiological Surveillance System used by physicians (SINAVE Med). The warning of these errors had already appeared in August, but only now has the full study been published.
Thus, the work analyzes the information transmitted by the General Directorate of Health (DGS) in two moments, a first shipment sent in April and a second in August. In the two data sets there are several errors and failures that, Cristina Costa Santos warns, can lead to erroneous conclusions about the evolution of the pandemic in Portugal and, worse still, to erroneous decisions in the response to this public health problem. There is, therefore, a warning and a call to be made: it is necessary that the 50 research groups that are handling the data provided by DGS on covid-19 know that these data have errors, inconsistencies and omissions and it is necessary to alert the authorities the ways that can and should be taken to correct the quality problem of these records.
Errors and incomplete data
There are several examples of errors in the SINAVE data article provided by DGS to researchers. “Probably the age of a patient is incorrect (134 years)”, reads at the beginning of a paragraph that mentions other problems: “There were three male patients and one older female (97 years) registered as pregnant. There is an incorrect diagnosis date (05-05-2020) and 19 patients had registered diagnosis dates before the first official case of covid-19 was diagnosed in Portugal. There were also two patients with a negative hospital stay. The variable ‘recovery date’ had only three records, despite referring to a period of 120 days: one for 6772 patients, where the recovery date was ‘April 3’, another for 1032 patients registered in ’25 de Maio ‘and also for 242 patients in whom the recovery date was recorded in’ 26 de Maio ‘”.
There are still more than four thousand cases in the April report that do not appear in the August document. On the other hand, there are obvious discrepancies between the SINAVE data and those published in the DGS daily bulletins. “No June death dates were reported in the DGS August dataset, despite 155 deaths reported in the public report during this month,” the article says.
Cristina Costa Santos highlights yet another type of problem: incomplete data. From the first set of data sent by DGS in April to the second set sent in August, it was concluded that, in 40% of patients, the answer to the question on pre-existing diseases “changed from ‘no’ to ‘I don’t know’” . What may have happened is that a first blank answer was initially interpreted as “no” and only later corrected to “I don’t know”, but having a patient without diabetes, for example, is not the same as not knowing if he has diabetes or not. Any team that is analyzing and drawing conclusions from these data will reach a biased result, warns the researcher from FMUP and the Center for Research in Health Technologies and Services (Cintesis).
The researcher points out that several teams of researchers wanted to get involved in the data collection and processing process from the beginning of the pandemic, however, this never happened. Therefore, he notes, the results “are not surprising.” It was already known that SINAVE’s existing database was far from perfect, but covid-19 brought to light a number of weaknesses in this system. “These systems have never worked well, they only served to defend against situations of notifiable diseases, but with the volume of data that we have at the moment, with thousands of cases, it was necessary to have an information system that worked.”
Cristina Costa Santos explains that doctors are called to fill out files on different platforms (there are at least three different ones), with numerous fields to fill in, when their priority is obviously to attend to the patient in front of them. “They can’t, it’s not possible,” he says. “We needed a system to help us and, for that reason, the academy offered to help,” reinforces Cristina Costa Santos. There are several examples of the bureaucratic workload required of clinicians in the article, with questions ranging from general demographic characteristics to questions related to individual symptoms, including 10 specific questions on comorbidities and more than 20 questions to characterize clinical outcomes. the severity of the illness and the use of healthcare resources, including details of hospital isolation.
In addition to the SINAVE used by doctors, there is also the SINAVE Lab, which serves laboratories, and also the Trace Covid-19 (a tool for monitoring the covid-19 pandemic, of patients under surveillance and self-care). The researchers only received (in April and then in August) the data from SINAVE Med. In the article, the FUMP team highlights that several questions were sent to DGS about the problems identified in the quality of the data. And Cristina Costa Santos confirms that, so far, they have not had an answer. “They always responded with great sympathy to our contacts, but we had no answer to the questions we asked. They tell us they have a lot of work and we understand that. However, these problems are not a detail. Having quality data is decisive for making good decisions ”.
In August, when these problems began to be reported by researchers, DGS acknowledged the missing data and justified that doctors did not have time to complete everything. At that time, in the face of criticism, a specialist from the DGS explained to the PUBLIC that the central objective of the epidemiological surveillance system is to allow “quickly blocking the transmission chains to contain the epidemic.” “This is the priority and we are in a completely abnormal situation, so the data validation work is less precise and less urgent,” he added. This Monday at a press conference, General Director Graça Freitas stressed that these SINAVE data provided to groups of scientists are epidemiological surveillance data and not scientific research data and, therefore, “we know they are not perfect.” The official also considered that “academics have a very important role in evaluating the quality of these data” and that researchers can make corrections and improve the quality of the information available.
Solutions
But are we still in time to correct mistakes or, at least, avoid more mistakes? Cristina Costa Santos thinks so. Now it is more complicated, but it is still possible to repair the damage. “This work is to show that the quality of the data is really bad, but also to say that we have to do something, we always have time to improve,” he says. Especially since, unfortunately, the covid-19 pandemic shows no signs of a truce.
“It is necessary to define and implement important improvements in surveillance data systems and processes: simplification of data entry processes, constant monitoring of data, awareness of health providers about the importance of having good data and adequate training. thereof. Data processing processes, leveraging effective, multidisciplinary collaborations between healthcare providers and data analysts, play a critical role in ensuring minimum quality standards. With these fully optimized processes, the reliability of the results and the quality of the scientific evidence produced can be greatly improved ”, suggest the authors of the article, who acknowledge that this data quality problem will not be unique to Portugal.
In fact, FMUP researchers even refer to another study that detected errors and discrepancies in the information on covid-19 after analyzing the databases of the World Health Organization, the European Center for the Control and Prevention of Diseases and the Center. Chinese for Disease Control and Prevention and which was also carried out by scientists from Portugal. Jorge Bravo and Afshin Ashofteh, authors of the article published in the magazine Statistical journal under the title “Study on the quality of the databases of the new coronavirus”, they found error margins higher than 40%, negative values and many other discrepancies in the databases.
The FMUP researcher requested data on covid-19 from DGS in order to analyze the risk factors associated with the most severe cases and mortality. The first step in his research work was precisely to confirm the quality of the data that, according to DGS, will have been sent to 50 research groups that requested them. And that’s when the project came to a halt and ended up being diverted to the now published article on “Covid-19 Surveillance – A descriptive study on data quality issues”, which Portugal uses as a case study. A bad example, he concludes.
