Abstracts

Introduction

Scientometrics is an information science that seeks to study primarily the quantitative aspects of science and scientific production ¹1. Ricker M. Letter to the Editor: about the quality and impact of scientific articles. Scientometrics 2017; 111:1851-5.^,22. Groneberg DA, Klingelhöfer D, Brüggmann D, Scutaru C, Fischer A, Quarcoo D. New quality and quantity indices in science (NewQIS): results of the first decade - project progress review. Scientometrics 2019; 121:451-78.^,33. Barreto ML. The challenge of assessing the impact of science beyond bibliometrics. Rev Saúde Pública 2013; 47:834-7.. Since the 1980s, Brazil has presented a noteworthy increase in absolute number of publications, particularly in the biomedical and health areas ⁴4. Guimarães JA. A pesquisa médica e biomédica no Brasil: comparações com o desempenho científico brasileiro e mundial. Ciênc Saúde Colet 2004; 9:303-27.. However, in recent years, it has occasionally been released to the general public, in daily or weekly newspapers and magazines, that this increase in productivity has led to a decrease in quality, as measured by mean citations per publication ⁵5. Moraes FT. Brasil aumenta produção científica, mas impacto dos trabalhos diminui. Folha de S.Paulo 2017; 16 oct. https://www1.folha.uol.com.br/ciencia/2017/10/1927163-brasil-aumenta-producao-cientifica-mas-impacto-dos-trabalhos-diminui.shtml.
https://www1.folha.uol.com.br/ciencia/20... ^,66. Leite R. Produção científica e lixo acadêmico no Brasil. Folha de S.Paulo 2015; 14 jan. http://www1.folha.uol.com.br/fsp/opiniao/202892-producao-cientifica-e-lixo-academico-no-brasil.shtml.
http://www1.folha.uol.com.br/fsp/opiniao... . Nonetheless, such fluctuating relation between quality and quantity has not yet been shown and, as such, an increase in the amount of publications should not be understood as an increase or decrease in overall quality.

Science has an exponential growth ⁷7. De Solla Price DJ. Science since Babylon: enlarged edition. New Haven: Yale University Press; 1975., doubling nearly every 10 to 15 years ⁸8. Fernández-Cano A, Torralbo M, Vallejo M. Reconsidering Price's model of scientific growth: an overview. Scientometrics 2004; 61:301-21.. This has been also described for Brazilian public health research output ⁹9. Nadanovsky P. Growth in Brazilian scientific output in public health dentistry. Cad Saúde Pública 2006; 22:886-7.^,1010. Celeste RK, Bastos JL, Faerstein E. Trends in the investigation of social determinants of health: selected themes and methods. Cad Saúde Pública 2011; 27:183-9.^,1111. Barros AJD. Produção científica em saúde coletiva: perfil dos periódicos e avaliação pela Capes. Rev Saúde Pública 2006; 40(n.spe):43-9. as well as the output of non-Brazilian journals ¹²12. Celeste RK, Broadbent JM, Moyses SJ. Half-century of dental public health research: bibliometric analysis of world scientific trends. Community Dent Oral Epidemiol 2016; 44:557-63.^,1313. Barreto ML. Crescimento e tendência da produção científica em epidemiologia no Brasil. Rev Saúde Pública 2006; 40(n.spe):79-85.. With such increasing trends in the absolute number of publications, there will be more papers cited. Indeed, as an example, there is evidence of an increase in mean citations of two traditional public health Brazilian journals between 1996 and 2005 ¹⁴14. Barata RB. SciELO Saúde Pública: o desempenho dos Cadernos de Saúde Pública e da Revista de Saúde Pública. Cad Saúde Pública 2007; 23:3031-40.. In Brazil, as in other countries, there are behaviors such as excessive self-citation and endogamy, with publishing in local journals representing about a quarter of all the production of the Web of Science (WoS) ¹⁵15. Collazo-Reyes F. Growth of the number of indexed journals of Latin America and the Caribbean: the effect on the impact of each country. Scientometrics 2014; 98:197-209.; furthermore, an increase in the volume of Brazilian journals in bibliographic databases can lead to an average increase in citations of articles by Brazilian authors. On the other hand, Brazilian public health is a rather specific area, which tends to deal with local and regional issues and will unlikely received citation from countries with different public health priorities and health system organization. Still, effective research and interventions in the context of rich countries may not work in the developing country either ¹⁶16. Miranda JJ, Zaman MJ. Exporting "failure": why research from rich countries may not benefit the developing world. Rev Saúde Pública 2010; 44:185-9..

Attached with the preceding problem, the scientific quality of a publication has not been clearly defined; often, it relies on subjective assessment made by other scholar about the originality, innovation, and methodological quality as components of the peer-review process ¹⁷17. Guyatt GH, Haynes RB. Preparing reports for publication and responding to reviewers' comments. J Clin Epidemiol 2006; 59:900-6.^,1818. Benos DJ, Kirk KL, Hall JE. How to review a paper. Adv Physiol Educ 2003; 27:47-52.^,1919. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 2009; 62:1013-20.. Regardless of how it is measured, high-quality publications are expected to have some impact. Although the social impact has been recognized as an important outcome of science ²⁰20. Smith R. Measuring the social impact of research: difficult but necessary. BMJ 2001; 323:528., there is no standard way to measure it. Nonetheless, the scientific impact of an article has been measured using citation counts ²¹21. Aksnes DW, Langfeldt L, Wouters P. Citations, citation indicators, and research quality: an overview of basic concepts and theories. SAGE Open 2019; 9:215824401982957.^,2222. Abramo G. Revisiting the scientometric conceptualization of impact and its measurement. J Informetr 2018; 12:590-7.. The use of citations of a paper as a proxy for its quality has been consistently criticized on several grounds ²¹21. Aksnes DW, Langfeldt L, Wouters P. Citations, citation indicators, and research quality: an overview of basic concepts and theories. SAGE Open 2019; 9:215824401982957.^,2323. Akerman M. Medidas de experiência e cienciometria para avaliar impacto da produção científica. Rev Saúde Pública 2013; 47:824-8., but the number of citations shows how much a work has influenced future research in a specific area of research.

While most traditional scientific journals attract the attention of expert readership, others try to get a larger audience of lay media, to engage people opinion, and to influence government and international organization policies ²⁴24. Tonetti MS. Leadership in publishing. J Dent 2019; 87:28-31.. Such journals are highly cited inside and outside the academic world, granting prestige to authors. New journals tend to prioritize more frequently the citation of articles from high-impact journals, published in English and from high-income countries. Importantly, the growth of publications in any database (Scopus or WoS) is both a reflex of an increase from existing journals and an increase from new journals indexed in the base ¹⁵15. Collazo-Reyes F. Growth of the number of indexed journals of Latin America and the Caribbean: the effect on the impact of each country. Scientometrics 2014; 98:197-209., with a new trend of open access journals - or prepaid publication with supplemented subscription-based access by self-archiving their own paper, to make it accessible free for all on the web ²⁵25. Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, Brody T, et al. Self-selected or mandated, open access increases citation impact for higher quality research. PLoS One 2010; 5:e13636.. Therefore, it is possible that high-prestige journals would increase their citations more than others over time.

Scientific Electronic Library Online (SciELO) is a Brazilian initiative to index open access online journals from Latin America and other countries with several of its journals being mostly indexed in Scopus ¹⁵15. Collazo-Reyes F. Growth of the number of indexed journals of Latin America and the Caribbean: the effect on the impact of each country. Scientometrics 2014; 98:197-209.. It is unknown whether the inclusion of SciELO journals in that database would influence the citation of other existing SciELO journals in the database. Furthermore, the Brazilian Graduate Studies Coordinating Board (CAPES) acts as a catalyst and inducer state agency for the creation of research capacity and the respective scientific publication in the country ²⁶26. Cross D, Thomson S, Sinclair A. Research in Brazil: a report for CAPES by Clarivate Analytics. http://www.capes.gov.br/images/stories/download/diversos/17012018-CAPES-InCitesReport-Final.pdf (acessado em Fev/2020).
http://www.capes.gov.br/images/stories/d... . Thus, it has long been used as a system to evaluate graduate programs, classifying them based on various indicators. The system includes the number of publications and the impact factor of the respective journals and is used to allocate resources, encouraging programs to publish more in high-impact journals. It is highly criticized for stimulating competitiveness, generating regional inequalities and establishing little communicative social connection, giving the population the wrong perception of a trivial impact on social reality ³3. Barreto ML. The challenge of assessing the impact of science beyond bibliometrics. Rev Saúde Pública 2013; 47:834-7.^,2727. Camargo Jr. KR. Produção científica: avaliação da qualidade ou ficção contábil? Cad Saúde Pública 2013; 29:1707-11..

Although it is a genuinely relevant issue for any country’s scientific policy, there is no consistent body of evidence describing trends and influences in the profile of Brazilian scientific publication, at least from the point of view of the injunctions discussed in this introduction. Therefore, this study has two objectives. First, to assess the percentage of articles with at least one author affiliated to Brazilian institutions in high-impact journals and SciELO, including public health journals. Second, to evaluate trends in 5-year citations according to the author’s affiliation and different journal categories.

Methods

The Scopus database was chosen because it contains a larger number of Latin American journals in its main collection than other databases. The analytical tools provided by this database were used as a source of citations, publications, and other bibliometric data. Data were extracted annually and separately from 1995 to 2019.

Journals selection

Journals were selected to compose four categories: (1) High-impact General Health journals; (2) High-impact Public Health journals; (3) SciELO General Health journals; and (4) SciELO Public Health journals. The Scimago Journal & Country Rank website (https://www.scimagojr.com) was used to find and select journals, as it contains information about journals indexed in the Scopus database (https://www.scopus.com). Journals were identified using specific filters for relevant subject areas and subcategories.

A common criterion to all groups was that the journals should have publications throughout the defined period. The scope of each selected journal was read to assess if it covers health-related areas and public health. Selection started with the journal with the highest H-index, adding more journals until the number of publications with authors affiliated to Brazilian institutions summed approximately 100 in 1995. The strategy to start each series with approximately 100 articles is arbitrary, but it has proven useful to avoid random variations over the years due to the presence a highly cited paper in one specific year. Specific criteria were applied to each group as follows:

For High-impact General Health (HGH), all journals from any subject area, with H-index > 200 were screened. There were 321 eligible journals in the year 1995. If a journal fitted Public Health subject category, it was relocated to Public Health. No SciELO journal was found among this group.

For High-impact Public Health (HPH), all journals with H-index > 75 were screened among the following subject areas: Medicine, Nursing and Social Sciences within the following subcategories: “Public Health, Environmental and Occupational Health”, “Epidemiology”, “Health policy”, “Infectious Diseases”, “Nutrition and Dietetics”, and “Health (Social Science)”. There were 196 eligible journals in the year 1995. It was not possible to apply the same H-index of general health journals due to the number of journals available and the small number of authors affiliated to Brazilian institutions among them. The addition of non-core journals, such as Infectious Disease, Pediatrics or Nutrition, was based on the observation that many of those journals include, in their scope, areas like Epidemiology or Public Health. Moreover, the aforementioned areas have close connection to public health issues and those journals publish a considerable share of public health articles.

For SciELO General Health (SGH) journals, we screened all journals from any subject area in the database (there is a SciELO filter at Scimago) and included all that met general inclusion criteria. In 1995, there was 136 journals eligible, many of them not Latin American or further discontinued.

For SciELO Public Health (SPH) journals, we screened all journals from any subject area in the database (there is a SciELO filter at Scimago) and included all that met general inclusion criteria. This group is not a mirror of SciELO Public Health collection because the SciELO collection is not entirely included in Scopus and some of those journals are not Latin American. In addition, because of small number of journals the met the criteria (n = 9), we created a second series from 2007 to 2014 (n = 26). Preliminary analysis showed that the shorter series yielded similar results, but no trend was clear because the series was too short. Data for the second series is available upon request to authors.

Articles selection and citation received

For the retrieval of numerical data, the articles published, and number of citations received, we searched all the records published from 1995 to 2019, in total, 20 years were analyzed for citations and 25 for percentage of articles affiliated to Brazil. For each year, we selected articles published in each of the four groups of journals and the number of citations granted in the current and in the following five years were counted using Scopus filters.

We included only original articles and reviews using the filter: “LIMIT-TO (DOCTYPE,“ar”) OR LIMIT-TO (DOCTYPE,“re”)”. After obtaining all articles, the number of articles affiliated to Brazil were selected using the filter: (LIMIT-TO (AFFILCOUNTRY, “Brazil”). To avoid including journals with similar names, all articles were selected using the filter for sources-ID of the journals on the Scopus website, as each journal has a unique number in the database. Data was added to a spreadsheet for final conference of other authors and a random data collection was redone as quality control.

Outcome variables

Two outcomes were studied. Mean citations per document was calculated dividing the total number of citations received up to five years after publication by all articles in a specific year. For example, there was 29,445 articles published in 1995 among high-impact journals, and those articles were grated 840,085 citations up to the year 2000, resulting in a rate of 28.5 cited per document in five years. The percentage of articles from authors affiliated to Brazilian institutions was calculated dividing the number of articles retrieved for Brazil, using the country filter divided by the total number of articles. For example, there was 29,445 articles published in 1995 among high-impact journals, of which 89 had at least one authors affiliated to a Brazilian institution, resulting in 0.3%.

Statistical analysis

Descriptive data was shown for trends of 5-year citations and percentage of articles from authors affiliated to Brazilian institutions by journal groups in graphical presentation. To obtain coefficients of trends and to account for time-related values, we used generalized estimation equation models (GEE) with one lag autoregressive correlation matrix (AR1). Trends of 5-year citations were modelled using Poisson regression with logarithmic link-function, having total citations as outcome variable and log-number of articles as offset. Trends in percentage of articles from authors affiliated to Brazilian institutions were modelled using binomial regression with logit link-function, having the number of articles from authors affiliated to Brazilian institutions as outcome the total number of articles as denominator.

Absolute increase in citations and in percentage points were obtained using reverse functions (invlogit and exponential) added to intercept and interaction terms. Comparisons of trends among journal groups were tested using interaction terms of year and journals, comparisons of trends between citations of articles from authors affiliated to Brazilian institutions and all articles was also tested with an interaction between authorship and year. The variable “year” was modelled as increments for every 10 years change. Statistical analysis was carried out using Stata MP 13.1 (https://www.stata.com).

Results

The same journals in each group were followed over the period, but presented variations in the number of articles published, as well as in 5-year citations. The full list with the journals’ names can be seen in the Supplementary Material (http://cadernos.ensp.fiocruz.br/static//arquivo/suppl-e00197820_1103.pdf). In the first year of the series, about 12.7% of articles in the HGH journals had no information about affiliation country, falling to 5% in the following and subsequent years. Similar pattern was observed in the other journal groups for the year of 1995, with articles without any affiliation summing 9.4% in HPH, 9.1% in SGH, and 20% in SPH. All groups kept the percentage of publication without affiliation address below 5% throughout the series after 1996.

The group of HGH comprised 33 journals that published 29,445 articles in 1995 and 17,734 articles in 2019, with affiliation to Brazil increasing from 0.3% to 1.5%. The group of HPH comprised 81 journals that published 7,934 articles in 1995 and 17,074 in 2019, with affiliation to Brazil increasing from 1% to 3%. The group of SPH comprised nine journals that published 350 articles in 1995 and 1060 in 2019, with affiliation to Brazil increasing from 47.4% to 71.9%. The group of SGH comprised 23 journals that published 1,137 articles in 1995 and 2,147 articles in 2019, with affiliation to Brazil increasing from 49.7% to 55.4%. There was an increase in percentage of articles with any author affiliated to Brazil over the period in all groups (Figure 1).

Over the period, there was a statistically significant (p < 0.01) increase in mean 5-year citations per document in all journal groups (Table 1; Figure 2) and the slope of increase was also statistically different among journal groups (p < 0.01). For articles from authors affiliated to Brazilian institutions, the mean values of 5-year citations per document in 1995 and 2014 were, respectively, 31.0 and 120.4 for HGH, the regression-based slope of increase for every 10 years was 32 citations per document (95%CI: 31.1-33.0). For HPH, between 1995 and 2014, such citations increased from 9.6 to 19.0, the regression-based slope of increase for every 10 years was 4.2 citations per document (95%CI: 3.8-4.6). For SGH, citations increased from 1.1 to 6.1, the regression-based slope of increase for every 10 years was 1.8 citations per document (95%CI: 1.6-1.9). For SPH, citations increased from 1.4 to 8.0, the regression-based slope of increase for every 10 years was 2.3 citations per document (95%CI: 2.1-2.5).

Regression-based slopes of increase in 5-year citations per document were higher for authors affiliated to Brazilian institutions than for articles with any other affiliation (Table 1). The largest difference in increment of mean 5-year citations was observed among HGH journals with slopes of 11.9 versus 32.0 citations for every 10 years (p < 0.01) for articles with any affiliation compared to authors affiliated to Brazilian institutions. The smallest difference in increase over 10 year was observed among HPH journals with slopes of 3.7 versus 4.2 citations (p = 0.05) for articles with any authors compared to authors affiliated to Brazilian institutions. The increase in citations was also higher for Brazilian articles among SGH and SPH journals (p < 0.01).

Discussion

The main findings of our study do not confirm the assumption made by some commentators, regarding the decreasing presence of Brazilian science, at least based on the citation count. Firstly, there was an increase in 5-year mean citations per document over the period in all journal groups, but it was much higher for articles from authors affiliated to Brazilian institutions in the HGH journals. Secondly, authors affiliated to Brazilian institutions participated in more publications in all journal groups, including High-impact (General Health and Public Health) journals, although still with a low percentage of publications (< 3%).

In terms of the global number of publications in the selected journal groups, the HGH group reduced the annual volume of publications by 39%, while HPH almost doubled (115% increment). A reduction among the former maybe because they adopted stricter editorial policies. Importantly, a tendency among editorial corporations leading the journals in the HGH group has been to link their names to other journals to receive their surplus of manuscripts. For example, The Nature group, The Lancet, and Science have about, respectively, 57, 18 and 5 other journals that carry out their names at the beginning, such as The Lancet Public Health, Nature Medicine, or Science Immunology. Nevertheless, the increase in HPH has been reported elsewhere ¹⁰10. Celeste RK, Bastos JL, Faerstein E. Trends in the investigation of social determinants of health: selected themes and methods. Cad Saúde Pública 2011; 27:183-9. and now, we showed that the Brazilian participation grew even more than average. Participation of authors affiliated to Brazilian institutions grew five times in percentage points in HGH journals and three times in absolute numbers, but differently, among HPH journals, the Brazilian participation tripled in percentage and increased more than six times in absolute numbers.

This general trend that Brazilian science is growing above average, detached from any bias that could be produced by the SciELO base ²⁸28. Xia J, Myers RL, Wilhoite SK. Multiple open access availability and citation impact. J Inf Sci 2011; 37:19-28.^,2929. Sotudeh H, Horri A. Countries positioning in open access journals system: an investigation of citation distribution patterns. Scientometrics 2009; 81:7-31., has already been reported elsewhere ²⁶26. Cross D, Thomson S, Sinclair A. Research in Brazil: a report for CAPES by Clarivate Analytics. http://www.capes.gov.br/images/stories/download/diversos/17012018-CAPES-InCitesReport-Final.pdf (acessado em Fev/2020).
http://www.capes.gov.br/images/stories/d... . Using bibliometrics to analyze papers published between 2011 and 2016, analysts identified productivity gains regarding the Brazilian research and science policy: up to the date of the report, Brazil was the 13th largest world producer of research publications and its production grew annually. In fact, official data from important research funding sources in Brazil (CAPES, Brazilian National Research Council - CNPq, Brazilian Funding Authority for Studies and Projects - FINEP, and the States Research Foundations) show a steady increase in investment from 2001 to 2014 ³⁰30. Dudziak E. Levantamento mostra quem financia a pesquisa no Brasil e na USP. Jornal da USP 2018; 26 jul. https://jornal.usp.br/ciencias/levantamento-mostra-quem-financia-a-pesquisa-no-brasil-e-na-usp/.
https://jornal.usp.br/ciencias/levantame... ^,3131. Moura EG, Camargo Jr. KR. A crise no financiamento da pesquisa e pós-graduação no Brasil. Cad Saúde Pública 2017; 33:e00052917.. Assuming that scientific growth may be linked to the corresponding financing, the austerity measures taken after 2015 will presumably have an impact on the trends reported here.

Analyzing reasons for the growth in the volume of scientific outputs in a country, especially articles published in journals, is not an idle but a complex task. Why is Brazilian science growing above average? One explanation for the increase in citations is that more Brazilian journals than average were added to the collections, and they tend to cite Brazilian articles, mainly from high-impact journals. CAPES has been criticized, as stimulating to the so called “salami publication” ³²32. Broad W. The publishing game: getting more for less. Science 1981; 211:1137-9.. Nonetheless, it remains to be shown that authors affiliated to Brazilian institutions are doing this more than other countries. A plausible explanation may be related to participation in international networks, as there is evidence that CAPES has promoted a strong internationalization process ³³33. Paiva FM, Brito SHA. O papel da avaliação CAPES no processo de internacionalização da pós-graduação em Educação no Brasil (2010-2016). Avaliação: Revista da Avaliação da Educação Superior (Campinas) 2019; 24:493-512.. It seems that international cooperation increases citations, as reported for Brazilian Space Science, but may not impact in the number of publications ²⁶26. Cross D, Thomson S, Sinclair A. Research in Brazil: a report for CAPES by Clarivate Analytics. http://www.capes.gov.br/images/stories/download/diversos/17012018-CAPES-InCitesReport-Final.pdf (acessado em Fev/2020).
http://www.capes.gov.br/images/stories/d... . Indeed, graduate programs level 6 or 7 (the highest levels in the country) must show international insertion, in addition to a strong academic output ¹¹11. Barros AJD. Produção científica em saúde coletiva: perfil dos periódicos e avaliação pela Capes. Rev Saúde Pública 2006; 40(n.spe):43-9.. We might speculate that Brazil started to have research problems of international interest in some strategic areas, but does this give it the qualification of world excellence? Is the observed grow a reflex of better quality of submitted manuscripts?

Another component of Brazilian scientific system policy, which helps to elucidate some research findings, is the role played by the CNPq. It is a national research agency that grants funds and scholarships to productive scientists to support their studies. A previous study investigating the scientific productivity of 323 researchers reported great variability in senior researchers in all sub-fields of the Biomedical Sciences ³⁴34. Kamdem JP, Roos DH, Sanmi AA, Calabró L, Abolaji AO, Oliveira CS, et al. Productivity of CNPq researchers from different fields in biomedical sciences: the need for objective bibliometric parameters - a report from Brazil. Sci Eng Ethics 2019; 25:1037-55.. The analysis of the most cited articles revealed that the researchers’ citations at junior levels were associated with publications with their supervisor, showing the importance of graduate programs. It is not easy to obtain information about quality and investments in Education, Science and Technology or any other measure taken to boost developmental research in any country ¹⁵15. Collazo-Reyes F. Growth of the number of indexed journals of Latin America and the Caribbean: the effect on the impact of each country. Scientometrics 2014; 98:197-209.. Even if it was possible, it may be difficult to compare such measures among countries (by policy, tax, distribution, legal system, or financing model adopted). A recent study showed that investments in basic education might increase scientific output at state level, although most of state level variations was explained by research structural and workforce factors (number of PhDs) ³⁵35. Celeste RK, Muniz FWMG, Ardenghi TM, Collares FM, Rösing CK. Brazilian dentistry research productivity. Braz J Oral Sci 2020; 19:e206977..

Our findings point to an increase in mean citations. In the Clarivate Analytics’ report for CAPES ²⁶26. Cross D, Thomson S, Sinclair A. Research in Brazil: a report for CAPES by Clarivate Analytics. http://www.capes.gov.br/images/stories/download/diversos/17012018-CAPES-InCitesReport-Final.pdf (acessado em Fev/2020).
http://www.capes.gov.br/images/stories/d... , citation in Brazil has historically been below the world average but has increased by more than 15% since 2012. The impact of the citation was computed as the normalized average citation count for a scientific publication or group of documents. Brazil produced some highly cited articles and achieved good paper rates in the top 1% of the world. Citation rates also depend on the area of research and the age of a scientific publication (older documents had more time to obtain citations compared to newer ones). However, how can Brazilian public health research stand on this scenario? The Clarivate Analytics’ report shows that Brazilian public health articles stand as second in total amount, but 16th in mean citations. Research on the Brazilian Unified National Health System (SUS), and tackling national priorities may explain this, as citations necessarily come from other Brazilian articles only.

Among various aspects that deserve to be discussed, there is the question of increasing the number of authors per article, which may increase the likelihood of authors affiliated to Brazilian institutions participating and may increase mean citations. It has been described an increase in median number of authors per articles, and proportion of articles with four or more authors in Brazilian journals of public health ³⁶36. Camargo Jr. KR, Coeli CM. Múltipla autoria: crescimento ou bolha inflacionária? Rev Saúde Pública 2012; 46:894-900.. Unfortunately, our study design cannot answer if this was a major driver behind our results, but it is good if authors affiliated to Brazilian institutions participate more in international collaborations. However, it would be questionable if it represents “honorary authorship”. The possibility of improper attribution of authorship is undoubtedly an ethical and editorial problem for reputable journals. On another front, it can also be difficult to control, as it can reflect the incentives to productivity generated by the Brazilian evaluation mechanisms, allowing the proliferation of authors to be a way of bypassing the evaluation system.

The increase in mean citations may hide a statistical problem of means, that is, the presence of outliers, also called “blockbusters”. Several academic works are barely mentioned, while others are highly cited. To summarize, three general categories were identified as related to the number of citations ³⁷37. Tahamtan I, Safipour Afshar A, Ahamdzadeh K. Factors affecting number of citations: a comprehensive review of the literature. Scientometrics 2016; 107:1195-225.: factors related to the article, to the journal and to the author(s). Most likely, some factors such as the quality of the article, the journal’s impact factor, the number of authors, visibility and international cooperation are stronger predictors of citations. The average citation per article is a consequence of many aspects, rather than only quality. As mentioned by Barreto ³3. Barreto ML. The challenge of assessing the impact of science beyond bibliometrics. Rev Saúde Pública 2013; 47:834-7., any new scientific knowledge comprises cumulative elements, as new projects must cite prior knowledge needed to support and to justify them. This is the basis of bibliographic cataloguing systems and, based on them, the citation count. Oftentimes, however, the high number of citations is not cognitively essential, but substitutional ¹1. Ricker M. Letter to the Editor: about the quality and impact of scientific articles. Scientometrics 2017; 111:1851-5..

Additionally, an important remark regards the controversy about the quality of Brazilian scientific publication, illustrated with some newspaper articles commenting on the (lack of) quality and its (in)efficiency. An answer to that ³⁸38. Silva V. Scientometrics: Nature Index and Brazilian science. An Acad Bras Ciênc 2016; 88:1597-9. shows arguments and sources of bias that, considering the dubio pro reo principle, make it problematic to question the quality and efficiency of Brazilian science for reasons mentioned in those newspaper articles. Again, the main problem becomes whatever indicators are chosen for scientometric analysis. In addition, there is a high risk of ecological fallacy in the comparison between countries, in relation to the efficiency in the use of the resources applied in their research effort. Obviously, if a country tries to maintain only a few relevant research groups, it can achieve many citations per article. Unacceptably, this may lead to the exclusion of Social Sciences that tends to receive fewer citations, regardless of scientific contribution or quality. The problem is the so-called Matthew Effect ³⁹39. Merton RK. The Matthew Effect in Science, II: cumulative advantage and the symbolism of intellectual property. Isis 1988; 79:606-23., a principle developed by Robert Merton in the Sociology of Science on the “social selection” process that leads to the concentration of scientific resources and stimuli among “elite” scientists, as well as the journals they publish that have a strong influence on their citation rates ⁴⁰40. Larivière V, Gingras Y. The impact factor's Matthew Effect: a natural experiment in bibliometrics. J Am Soc Inf Sci Technol 2010; 61:424-7.. On the other hand, the Matthew Effect changes if a country has a more distributed model of research funding. It seems evident that the more you invest in Science and Technology in a democratic and equitable way for different groups and institutions/regions, more space is opened for the publication of articles that, on average, may have fewer citations individually, but add up to a total greater number of citations.

Unfortunately, little has been done to systematically assess the social impacts of different scientific fields on social life, in addition to the bibliographic effects and scientific impact. Bibliometrics allows measuring only part of science (publication and citations), leaving aside its various non-bibliographic effects on the world and the real life of populations. In Brazil, this process took on stressed features, as it is associated with the development of postgraduate courses at universities, and bibliometric indicators were enthusiastically incorporated into the system for evaluating the scientific and academic activities of researchers ³3. Barreto ML. The challenge of assessing the impact of science beyond bibliometrics. Rev Saúde Pública 2013; 47:834-7.. The focus on the number of articles, impact factor or journal H-index has entailed criticism recently, as it has disadvantages and is unsatisfactory ⁴¹41. Iriart JAB, Deslandes SF, Martin D, Camargo Jr. KR, Carvalho MS, Coeli CM. A avaliação da produção científica nas subáreas da Saúde Coletiva: limites do atual modelo e contribuições para o debate. Cad Saúde Pública 2015; 31:2137-47.. It has also been recurrently criticized that the quality of teaching and training of graduate students are undervalued ⁴²42. Spagnolo F, Calhau MG. Observadores internacionais avaliam a avaliação da CAPES. Infocapes: Boletim Informativo 2002; 10:5-33.^,4343. Barata RB. Necessary changes in the evaluation of graduate programs in Brazil. Interface (Botucatu) 2019; 23:e180635.. We expect that, at some point, this knowledge will be transformed into action, generating tangible benefits (material or non-material) for society, in the process currently known as knowledge transfer. There is a need for substantial changes, to stimulate solidarity ⁴¹41. Iriart JAB, Deslandes SF, Martin D, Camargo Jr. KR, Carvalho MS, Coeli CM. A avaliação da produção científica nas subáreas da Saúde Coletiva: limites do atual modelo e contribuições para o debate. Cad Saúde Pública 2015; 31:2137-47., including other evaluative and inductive parameters, such as practical applications and innovations produced by scientific activity to increase social well-being and reduce the burden of poverty, disease and suffering in large segments of societies.

Acknowledgments

R. K. Celeste has a PQ2 fellowship from the Brazilian National Research Council (CNPq: 311592/2019-8).

References

Publication Dates

History