Meta-analysis: Problems with Russian Publications

BACKGROUND:

Meta-analysis is a powerful tool to identify Evidence Based medical technologies (interventions) for use in every day practice. Meta-analysis uses statistical approaches to combine results from multiple studies in an effort to increase power (over individual studies), improve estimates of the size of the effect and/or to resolve uncertainty when reports disagree. Meta-analysis is a quantitative, formal study design used to systematically assess previous research studies to derive conclusions from this research. Meta-analysis may provide more precise estimate of the effect of treatment or risk factor for a disease, or other outcomes, than any individual study contributing to the pooled analysis.

We have quite a substantial number of Russian medical publications, but not so many Meta-Analyses published in Russian. Russian publications are cited in English language papers not so often. A total of 90% of clinical studies included in published Meta-Analyses incorporate only English language papers. International studies or papers with Russian co-authors are published in English language.

OBJECTIVE:

The main question is: what is the problem with inclusion of Russian medical publications in Meta-Analysis?

RESULTS:

The main reasons for this are the following:

1) It is difficult to find Russian papers, difficult to work with them and to work with Russian journals:

a. There are single Russian Biomedical Journals, which are translated into English and are included in databases (PubMed, Scopus and other), despite the fact that all of them have English language abstracts.

b. The majority the meta-analyses authors use in their work different citation management software such as the Mendeley, Reference Manager, ProCite, EndNote, and others. These citation management systems allow scientists to organize their own literature databases with internet searches and have adds-on for the Office programs what makes process of literature citation very convenient. The Internet sites of the majority of International Journals have built-in tools for saving citations to reference manager software. The majority of articles in Russian journals cannot be captured by citation management systems: they do not have special coding of articles descriptors.

c. Some journals still have PDF files of the whole journal issue without dividing it into articles and do not provide any descriptors, making manual time-consuming input of information the only possibility. Moreover the context search of the article content is unavailable for search engines.

2) The quality of research. This problem has been discussed for more than twenty years already. Still we have too many publications of poor quality of study design and statistical analysis. With the exception of pharmacological clinical tails, designed and supervised by international Pharma industry, many interventional studies, conducted in Russia, have methodological flaws inferring a high risk of bias:

a. Absence of adequate control,

b. No standard endpoints, duration of therapy and follow up,

c. Absence of randomization and blinding,

d. Low power of studies: sample sizes are calculated (if calculated at all) in such a way, that the main goal is to have as small sample size as possible. Very often statisticians have to solve the problem how to substantiate a small number of subjects, that sponsor could afford, instead of calculating the needed sample size to reach enough power.

e. No standards of statistical analysis.

f. Russian journals do not have standards for description and presentation of study results, in particular, results of statistical analysis (a reader even cannot see what is presented: standard deviation (SD) or standard error of the mean (SEM).

We have a long standing experience in analysis of methodological and statistical quality of Russian biomedical publications and have found up to 80% publications with statistical and methodological errors and high risk of bias.

In our practice, we had tried to perform two Meta-analyses for two local pharmaceutical products for prevention of stroke recurrence. For the first product, we did not found even two single Russian language studies suitable for the analysis (incomparable populations, different designs, endpoints, doses etc.). For the second product, only four studies had comparable populations and standard internationally approved scales for effectiveness analysis. However, the combinations of scales, the length of treatment and follow up differed widely, so that we could combine the results of only 2 or 3 studies for each end point.

CONCLUSIONS:

Russian researchers have to follow internationally recognised standards in study design, selection of endpoint, timelines and therapy regimens, data analysis and presentation of results. Russian journals need to develop consolidate rules for authors of clinical trials and epidemiological research of result reporting close to international standards. In this case the international Network EQUATOR (Enhancing the QUAlity and Transparency Of health Research http://www.equator-network.org/) is one to be taken into account. In addition, Russian Journals have to improve their online information for better interaction with search engines and citation managers.