Assessment of the quality of health information on the Internet: evidence-based accuracy indicators for tuberculosis

Rodolfo Paolucci André Pereira Neto Paulo Nadanovsky About the authors

ABSTRACT

Not long ago, someone had to buy a newspaper, a book, or a magazine or go to a library to obtain information. Today, the Internet quickly facilitates a myriad of information. However, the information provided may be obsolete, incomplete, incorrect, or deliberately false: fake news. In the health field, this information can affect well-being or harm individuals and society. Thus, professionals, researchers, and institutions have assessed the quality of information on health websites to address this issue. Evaluations often verify the accuracy of the information provided. However, the information accuracy indicators have yet to be constructed from Evidence-Based Medicine (EBM). This article aims to build indicators from EBM practices, analyzing the case of tuberculosis. This manuscript proposes 43 information accuracy indicators that evaluated the tuberculosis information available on the Brazilian Ministry of Health. The results indicate that much information needs to be included, and some data must be corrected. This evaluation reiterates the importance of building EBM accuracy indicators. This work intends to encourage new studies about assessing the quality of health information on the Internet.

KEYWORDS
Evidence-based medicine; Internet; Access to information; Indicators (statistics); Tuberculosis

Introduction

Not long ago, someone had to buy a newspaper, a book, or a magazine or go to a distant, not very accessible library to obtain information. Letters were sent by post, and took some time to reach their final destination. Today, the Internet offers new opportunities for access and production of information11 Pereira Neto A, Barbosa L, Flynn M. Prefácio da edição brasileira – Há décadas em que nada acontece. Há semanas em que décadas acontecem. In: Pereira Neto A, Flynn M, organizadores. Internet e saúde no Brasil: desafios e tendências. 1. ed. São Paulo: Cultura Acadêmica; 2021. p. 7-18.. Anyone can access, produce, and share the information previously restricted to certain sociocultural groups22 Pereira Neto A, Paolucci R. Avaliação da qualidade da informação de saúde na internet: análise das iniciativas brasileiras. In: Pereira Neto A, Flynn M, organizadores. Internet e saúde no Brasil: desafios e tendências. 1. ed. São Paulo: Cultura Acadêmica; 2021. p. 257-291.. To this end, it is necessary to acquire an electronic communication device, to have technological conditions of access to the network, and to have the ability to handle these tools. For example, information can be accessed, produced, and shared on a website, blog, or Facebook profile. The volume of information available on the Internet is countless and covers any subject32 Pereira Neto A, Paolucci R. Avaliação da qualidade da informação de saúde na internet: análise das iniciativas brasileiras. In: Pereira Neto A, Flynn M, organizadores. Internet e saúde no Brasil: desafios e tendências. 1. ed. São Paulo: Cultura Acadêmica; 2021. p. 257-291.. Citizens can have all kinds of information at their disposal very quickly. Much information would only be accessed at the same speed44 Cheng C, Dunn M. Health literacy and the Internet: a study on the readability of Australian online health information. Aust New Zeal J Public Heal. 2015; 39(4):309-14. with the Internet. Digital media are increasingly pervasive and ubiquitous, reaching the point of building ‘smart cities’ and marking their presence in all things that surround and serve us55 João BN, Souza CL, Serravo FA. A systematic review of smart cities and the internet of things as a research topic. Cad EBAPEBR. 2020; 17(4):1115-30..

Health stands out as one of the areas with a growing volume of information available to an increasing number of interested people. The national survey coordinated by the Brazilian Internet Steering Committee (CGI.br) indicates that 45% of Internet users searched for information related to health or health services66 Martinhão MS. Pesquisa sobre o uso das tecnologias de informação e comunicação nos domicílios brasileiros: TIC domicílios 2018. São Paulo: Comitê Gestor da Internet no Brasil; 2019. [acesso em 2022 out 5]. Disponível em: https://www.cgi.br/media/docs/publicacoes/2/12225320191028-tic_dom_2018_livro_eletronico.pdf.
https://www.cgi.br/media/docs/publicacoe...
. This type of information has remained Brazilians’ second-largest information search activity, behind only information about products and services. In 2020, the total number of Brazilian Internet users who sought this type of information reached 53%77 Barbosa AF. Pesquisa sobre o uso das Tecnologias de Informação e Comunicação nos domicílios brasileiros: TIC Domicílios 2020: edição COVID-19: metodologia adaptada. São Paulo: Comitê Gestor da Internet no Brasil; 2021. [acesso em 2022 out 5]. Disponível em: https://www.cgi.br/media/docs/publicacoes/2/20211124201233/tic_domicilios_2020_livro_eletronico.pdf.
https://www.cgi.br/media/docs/publicacoe...
.

Access to quality health information on the Internet can positively affect health system managers88 Park H, Lee S-I, Hwang H, et al. Can a health information exchange save healthcare costs? Evidence from a pilot program in South Korea. Int J Med Inf. 2015; 84(9):658-66.,99 Spoelman WA, Bonten TN, Waal MWM, et al. Effect of an evidence-based website on healthcare usage: an interrupted time-series study. BMJ Open. 2016; 6(11). and citizens1010 Eysenbach G. The impact of the Internet on cancer outcomes. CA-A CANCER J Clin. 2003; 53(6):356-71.. Quality information promotes health promotion, as it facilitates the development of skills that give citizens greater decision-making power over their health and self-care1111 Garbin H, Guilam M, Pereira Neto A. Internet na promoção da saúde: um instrumento para o desenvolvimento de habilidades pessoais e sociais. Physis. 2012; 22(1):347-363.,1212 Pereira Neto A, Ribeiro B, Guljor AP, et al. Eu quero entrar na rede: análise de uma experiência de inclusão digital com usuários do Caps. Saúde debate. 2020; (44):58-69.. It can also interfere with the traditional asymmetrical doctor-patient relationship1313 Garbin HB, Pereira Neto A, Guilam MCR. A internet, o paciente expert e a prática médica: uma análise bibliográfica. Interface – Comum. Saúde, Educ. 2008; (26):579-88.. On the other hand, incorrect, incomprehensible, or outdated information can trigger decision-making harmful to health. Therefore, access to low-quality information can adversely affect the health of citizens.

In this context, Lemos1414 Lemos A. Cidade e mobilidade. Telefones celulares, funções pós-massivas e territórios informacionais. MATRIZes. 2007; 1(1):121-37. makes a distinction between mass and post-mass media. The first has a centralized flow of information, where firms control the information to serve their financing agents. They play an important social and political role in shaping public opinion. The information is addressed to people in an undifferentiated way, with little interacting possibility. In turn, post-mass media work from networks where anyone can produce information. In this case, there is neither a content producer pole nor dependence on advertising funds. The product is customizable and multidirectional, aimed at particular niches that translate specific interests. These conditions facilitate the flourishing outdated, incomplete, incorrect, or deliberately lying posts: fake news.

Oliveira1515 Oliveira T. Como enfrentar a desinformação científica? Desafios sociais, políticos e jurídicos intensificados no contexto da pandemia. Liinc em Revista. 2020; (16):e5374. admits three possibilities for addressing disinformation. In his view, this can be accomplished through fact-checking tools. The author also understands that citizens have the competency to make rational decisions based on their search for information. She suggests media and information literacy. Given the amount of information disseminated on a large scale in digital media, quality assessment can be considered another possibility to tackle fake news. It seems imperative to us vis-à-vis websites linked to public institutions.

In the last decade, researchers evaluating the quality of health information on the Internet have advanced in producing knowledge on how to deal with this issue. In a systematic review1616 Paolucci R, Neto AP. Methods for evaluating the quality of information on health websites: Systematic Review (2001-2014). Lat Am J Dev. 2021; 3(3):994-1056. on evaluation methods, no study used Evidence-Based Medicine (EBM) practices for developing indicators for the criterion of information accuracy: “a criterion to assess the conformity of information with the best and most current scientific evidence available”1616 Paolucci R, Neto AP. Methods for evaluating the quality of information on health websites: Systematic Review (2001-2014). Lat Am J Dev. 2021; 3(3):994-1056.(159). In these studies, as analyzed in two previous reviews1717 Eysenbach G, Powell J, Kuss O, et al. Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web. JAMA. 2002; 287(20):2691-700.,1818 Zhang Y, Sun Y, Xie B. Quality of health information for consumers on the web: A systematic review of indicators, criteria, tools, and evaluation results. J Assoc Inf Sci Technol. 2015; 66(10):2071-84., the information accuracy indicators were constructed through the consensus of experts, scientific and technical manuals, medical guidelines, textbooks, or literature. Paolucci, Pereira Neto, and Nadanovsky1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88. developed a set of methods to fill this gap.

This article aims to build evidence-based indicators addressing the specific case of tuberculosis. Tuberculosis was chosen due to its high incidence and mortality in Brazil, especially among the poor population living in urban areas2020 Brasil. Ministério da Saúde, Secretaria de Vigilância em Saúde. Boletim Epid. 2019 [acesso em 2018 jan 13]; (9). Disponível em: http://portalarquivos2.saude.gov.br/images/pdf/2019/marco/22/2019-009.pdf.
http://portalarquivos2.saude.gov.br/imag...
.

Material and methods

We employed the methods developed by Paolucci, Pereira Neto, and Nadanovsky1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88. to build indicators. They propose that information accuracy indicators be based on the best available scientific evidence. This article followed the seven-step set of methods: search strategy, evidence-based information source selection, topic collection and selection, development of the first version of indicators, group analysis, topic analysis, and indicator analysis.

Developing information accuracy indicators

In the first step, carried out on April 17, 2019, we applied the search strategy using the word “tuberculosis” in the ACCESSSS2121 McMaster U. About. 2019. [acesso em 2019 maio 30]. Disponível em: https://www.accessss.org/Pages/About.
https://www.accessss.org/Pages/About...
meta-search service, which returned 140 results in the fourth level of organization of evidence distributed in three sources of information.

In the second step, we selected DynaMed Plus2222 DynaMed. About DynaMed Clinical Decision Support Tool. 2019. [acesso em 2019 maio 7]. Disponível em: https://www.dynamed.com/home/about.
https://www.dynamed.com/home/about...
as a source of information to develop the indicators as it currently offers evidence-based guidance1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.. We also considered other similar platforms for collecting adequate information to construct accuracy indicators when making this option. A recent study1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88. compared sources of information similar to DynaMed Plus. In this study, we used DynaMed Plus, as this platform provides synthesized summaries for clinical reference. It stands out for its quality and way of presenting the results1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88., and we also managed to narrow down the search results to 40.

In the third stage, we collected and selected TB-related topics. We applied the exclusion criteria proposed by the authors of the method on topics addressing specific drugs and tests, meta topics, or topics without information1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.. Thus, we arrived at a total of 20 topics included in our sample.

In the fourth step, we read the topics and wrote the initial version of the indicators. Next, we classified the indicators into groups representing issues relevant to tuberculosis. As a result, we developed the first version of 180 information accuracy indicators (annex 1). However, we considered this number of indicators too extensive to be applied in evaluating health sites aimed at users.

In the fifth step, the group analysis allowed gathering of all indicators in at least one of the dimensions we specified for tuberculosis: prevention, transmission, symptoms, diagnosis, and treatment. In this analysis, we identified 27 indicators that needed to be grouped. The total number of indicators by dimension was as follows: prevention (30), transmission (22), symptoms (22), diagnosis (54), and treatment (60). Some indicators were classified into two dimensions simultaneously. This overview of the number of indicators in each group and possible intersections between them allowed us to reflect on how the sample could be circumscribed to meet the research objectives. It is necessary to shortlist the number of indicators to a feasible set containing essential information to evaluate websites on TB aimed at users. Furthermore, the purpose of health site reviews is not to gather all available evidence on tuberculosis. In this regard, we decided to circumscribe the selected sample of topics found in DynaMed Plus. We, therefore, returned to ponder on the topics included.

In the sixth step, we applied an exclusion criterion in the analysis of topics following the ideas proposed by the authors of the method1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.. The criterion is related to the incidence of different types of TB. This information was found on the service pages themselves. We can include the types of tuberculosis in the pulmonary and extrapulmonary groups.

Pulmonary TB is estimated to have reached ten million people and caused 1.6 million deaths worldwide in 2017 alone2323 DynaMed. Record No. T116300. Pulmonary Tuberculosis. 2018. [ acesso em 2019 jul 23]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T116300.
https://www.dynamed.com/topics/dmp~AN~T1...
. Regarding extrapulmonary TB, we have the following data: abdominal tuberculosis accounts for 5% of cases of extrapulmonary TB; bone and joint, for 10% of cases; disseminated or miliary, for 20% of cases; genitourinary, for about 5-6% of cases; and TB lymphadenitis, for 35% of cases2424 Paolucci R. Avaliação da qualidade da informação em sites de saúde: indicadores de acurácia baseada em evidência para tuberculose. [tese]. Rio de Janeiro: Fundação Oswaldo Cruz; 2020. 149 p.. Given the higher incidence of pulmonary TB than other forms of the disease, we excluded 99 indicators that contained specific information on extrapulmonary TB and reached a total of 81 indicators.

In the seventh and final stage, we analyzed the included indicators. We evaluated the wordings of the initial version as they were written almost as translations of the original clinical texts found in DynaMed Plus. The wording of the indicator should be intelligible to health professionals and users. Thus, they will be able to observe their presence or not on health websites1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.. In this analysis, we applied three methods of reformulating, gathering, and excluding indicators, as guided by Paolucci, Pereira Neto, and Nadanovsky1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88..

First, the three exclusion criteria proposed by the authors were used: technical information; considerations about diagnoses or treatments; and information intended for professionals or managers of health systems. The application of these criteria excluded 26 indicators, reducing the total to 55.

Then, we combined indicators with similar or complementary contents. The gathering was made with the information included in more than one indicator and by developing a new indicator wording based on information from two or three indicators. These methods reduced the number of indicators by twelve, totaling 43 information accuracy indicators in the final sample (table 1).

Table 1
Final version of the 43 ‘accuracy’ indicators for tuberculosis

Finally, we gathered content differently. However, it did not reduce the total because our analysis identified that the indicators involved were better written by transferring a piece of evidence from one indicator to the other. This procedure was performed in only one case.

Website selection

We arbitrarily selected a website to apply the developed indicators1616 Paolucci R, Neto AP. Methods for evaluating the quality of information on health websites: Systematic Review (2001-2014). Lat Am J Dev. 2021; 3(3):994-1056.. The chosen one is hosted on the Ministry of Health (MS) portal and is a glossary called ‘Health from A to Z’2525 Brasil. Ministério da Saúde. Saúde de A a Z – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-de-a-a-z.
https://www.gov.br/saude/pt-br/assuntos/...
. The pages of this glossary provide official information on health approved by the Brazilian Federal Government. For this reason, they are highlighted in Google Search and easily accessible by managers, health professionals, and users. This MS portal can be considered the main information reference for all state and municipal health secretariats. The information available it contains can be considered reliable. We located the topic ‘Tuberculosis’2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
under the letter ‘t’ of the glossary. So, we decided to evaluate the information available on this topic with the developed indicators.

Applying the indicators

Our proposed procedure for applying the accuracy indicators uses the same scale across the board: ‘incorrect’, ‘not found’, ‘incomplete’, and ‘complete’. This scale represents our judgment on the information found on the site when comparing it with the information from the indicators developed. The wording of the indicators was designed to contain all essential and evidence-based information that allows for assessing the quality of the information contained in a website.

Thus, both health professionals and users can evaluate the information. The idea is that the evaluator checks whether the indicator is available on the website and how it is arranged. If the information is not found, the evaluator should consider it ‘not found’.

If it is found but is wrong, the evaluator should consider it ‘incorrect’. Where the information is partial, the evaluator should consider it ‘incomplete’. However, if any part of the information contained in the indicator is wrong, even if another part is correct, we recommend that the evaluator deems it ‘incorrect’ for the indicator. Information should only be assessed as ‘complete’ if found and is correct as per the indicator. We employed this procedure on the selected page on February 24, 2022.

Results

Figure 1 presents the results of the seven stages1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88. for developing information accuracy indicators. We developed 43 indicators to assess the quality of information on tuberculosis for Internet users (table 1). We defined dimensions for tuberculosis to show how they were organized: prevention, transmission, symptoms, diagnosis, and treatment. The indicators are organized into groups within each dimension.

Figure 1
Methodological flowchart

The prevention dimension contains 16 indicators. It holds the most significant number of indicators, evenly distributed into four groups. The ‘BCG’ group contains information related to the Bacillus Calmette-Guérin (BCG) vaccine as its main subject. The ‘HIV’ group contains indicators with this virus as the main related subject. The group ‘Child guardian’ contains information on preventive actions when those responsible for the children have had contact with TB. The main subject in the fourth group is ‘Children’.

The ‘transmission’ dimension contains six information accuracy indicators organized into five groups. There is only one indicator per group, except the group involving the population of ‘Children’. The ‘General’ group does not contain indicators based on specific information about a population, health condition, or intervention. Indicators in this group include information intended for the general population. This group was also created for all the next dimensions we defined. The group ‘General risk factors’ includes information intended for the general population. The group ‘Risk factors and HIV’ contains indicators of factors specific to people with HIV. The group ‘Risk Factors and Children’ contains two information accuracy indicators for this population. The fifth group, ‘Risk Factors and Resistant Tuberculosis’, is based on information on factors related to drug-resistant types of tuberculosis.

The ‘symptoms’ dimension contains two indicators classified into two groups: ‘General’ and ‘Children’. The ‘TB diagnosis’ dimension contains eight indicators organized into four groups, each with two indicators. The first group is ‘General’, the second is ‘HIV’, and the third is ‘Children’. The fourth group, ‘Resistant Tuberculosis’ contains indicators based on information related to the diagnosis of drugresistant tuberculosis.

The ‘TB treatment’ dimension contains eleven indicators categorized into four groups. The first group, ‘General’, contains three indicators. The second group was classified as ‘HIV’ and contained two indicators. The ‘Resistant Tuberculosis’ group contains five indicators. The fourth group, ‘Side effects’, contains an indicator based on information whose main subject is the possible side effects resulting from a treatment.

We used the procedure for applying the 43 information accuracy indicators on the TB page in the ‘Health from A to Z’ glossary of the MS2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
portal. The information was considered ‘incorrect’ in one indicator, ‘not found’ in 29, ‘incomplete’ in eleven, and ‘complete’ in only two indicators, which reveals poor compliance. Much evidence-based information could be available on this site entirely and correctly. Most of the information was not found or needed to be completed. Also, some information was found to need to be corrected. The complete evaluation result is shown in table 2.

Table 2
Evaluation of the topic ‘Tuberculosis’ in the MS glossary

Discussion

We highlight four cases that represent the existence of ‘complete’, ‘incomplete’, ‘not found’, and ‘incorrect’ information on the evaluated page.

The first is from the indicator developed within the TB transmission dimension that belongs to the ‘General’ group: ‘Tuberculosis is transmitted by air from one person to another when bacteria are expelled as aerosols by someone with pulmonary tuberculosis’ (Id 131, table 2). When investigating the glossary page, the information found is complete despite being written differently. The evidence supporting this indicator’s development refers to two book chapters2323 DynaMed. Record No. T116300. Pulmonary Tuberculosis. 2018. [ acesso em 2019 jul 23]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T116300.
https://www.dynamed.com/topics/dmp~AN~T1...
. The first is from a manual of clinical microbiology in its tenth edition2727 Pfyffer G, Palicova F. Mycobacterium: General characteristics, laboratory detection, and staining procedures. In: Versalovic J, Carroll K, Funke G, et al., editores. Manual of Clinical Microbiology. 10. ed. Washington: American Society for Microbiology Press; 2011. p. 472-502.. The second is from a book that is in its eighth edition2828 Fitzgerald D, Sterling T, Haas D. Mycobacterium tuberculosis. In: Bennett J, Dolin RBM, editores. Mandell, Douglas, and Bennett’s Principles and Practice of Infectious Diseases. 8. ed. New York: Saunders; 2015. p. 2787-2818.. Both references are traditional teaching textbooks and contain consolidated knowledge. They can also be considered outdated because the time required for their production cannot keep up with advances in medical knowledge2929 Burwell DS. Acquiring the evidence: How to find current best evidence and have current best evidence find us. In: Straus SE, Glasziou P, Richardson WS, et al., editores. Evidence-based medicine: how to practice and teach EBM. 5. ed. Elsevier Health Sciences; 2019. p. 57-90.. However, they are referred to as the best evidence in this case. Information on the ‘transmission of tuberculosis’ can be considered consolidated knowledge or has undergone a few changes over the years. In this sense, it is traditionally known information that the glossary makes available in full.

The second case concerns the indicator developed within the ‘TB transmission’ dimension that is part of the group ‘Risk factors and

HIV’: HIV is the most critical risk factor for tuberculosis. People with HIV are 20 to 30 times more likely to develop tuberculosis than HIV-negative people. Risk factors include: (1) residence in tuberculosis-endemic regions, (2) close contact with tuberculosis patients, (3) crowded housing (including incarceration), (4) poor ventilation in the home or work environments, (5) malnutrition, and (6) limited access to quality health care. (Id 93, table 2). We found the following in the glossary:

People Living with HIV (PLHIV)

Tuberculosis in people living with HIV is one of the conditions with the most significant impact on HIV and TB mortality in the country. These people are at greater risk of developing tuberculosis and are often only diagnosed with HIV infection during the investigation/confirmation of TB2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
.

In this case, we considered the glossary information ‘incomplete’. The evidence2929 Burwell DS. Acquiring the evidence: How to find current best evidence and have current best evidence find us. In: Straus SE, Glasziou P, Richardson WS, et al., editores. Evidence-based medicine: how to practice and teach EBM. 5. ed. Elsevier Health Sciences; 2019. p. 57-90. that supported the construction of this indicator refers to a review that investigated the state-ofthe- art knowledge about the HIV-TB relationship3030 DynaMed. Record No. T909352, Latent Tuberculosis Infection in Patients With HIV. 2019. [acesso em 2019 jul 22]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T909352.
https://www.dynamed.com/topics/dmp~AN~T9...
. The authors define this relationship as a syndemic, i.e., “convergence of two or more diseases that act synergistically to increase the burden of disease”3131 Kwan C, Ernst JD. HIV and tuberculosis: A deadly human syndemic. Clin Microbiol Rev. 2011; 24(2):351- 76.(352). It has had lethal consequences globally. The MS glossary only reports that HIV is “one of the conditions with the greatest impact on mortality”2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
. This information is close to the indicator’s assertion that HIV is the most important risk factor for TB. However, there is a relevant difference: HIV is highlighted in DynaMed Plus, which gives greater centrality to the relationship between the two diseases, a relationship that is currently considered inseparable3030 DynaMed. Record No. T909352, Latent Tuberculosis Infection in Patients With HIV. 2019. [acesso em 2019 jul 22]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T909352.
https://www.dynamed.com/topics/dmp~AN~T9...
. The same can be said about developing the disease. The MS glossary states that people with HIV are at greater risk of developing TB2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
. The indicator contains information that this population is 20 to 30 times more likely to develop TB than people without HIV (Id 93, table 2). The indicator provides accurate and quantified risk information, highlighting the problem of this interaction between HIV and TB. Moreover, only some risk factors contained in the indicator are available in the glossary. Therefore, we assessed this information as ‘incomplete’.

The third case is the indicator built within the TB prevention dimension and the ‘BCG’ group, which states that ‘Babies should not be vaccinated with BCG in the following two cases: suspected HIV infection; or born to women with HIV’ (Id 26, table 2). When investigating the glossary page2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
, we did not find any information about vaccine contraindications. We considered this information to be ‘not found’ in the glossary. The evidence3232 DynaMed. Record No. T905489, Bacille Calmette- Guerin (BCG) Vaccine. 2018. [acesso em 2019 jul 18]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T905489.
https://www.dynamed.com/topics/dmp~AN~T9...
that supported the development of this indicator refers to two publications linked to the World Health Organization (WHO). The first is a 2007 publication of its weekly epidemiological bulletin3333 World Health Organization. Revised BCG vaccination guidelines for infants at risk for HIV infection. Wkly Epidemiol Rec. 2007; 82(21):193-6.. The second document was published in 2015 by one of the prominent international organizations fighting tuberculosis: Stop TB Partnership3434 Stop TB. Partnership. The global plan to stop TB 2011- 2015: Transforming the fight towards elimination of tuberculosis. 2011 [acesso em 2022 out 5]. Disponível em: http://www.stoptb.org/assets/documents/global/plan/TB_GlobalPlanToStopTB2011-2015.pdf.
http://www.stoptb.org/assets/documents/g...
. The first evidence reference is a review of the BCG vaccine guidelines for children at risk for HIV3333 World Health Organization. Revised BCG vaccination guidelines for infants at risk for HIV infection. Wkly Epidemiol Rec. 2007; 82(21):193-6.. The document states that the review originated from the Global Advisory Committee on Vaccine Safety (GACVS) after analyzing more recent data at the time. This review refers to two studies that produced evidence of contraindication for BCG. Published in 2005, the first analyzed the late complications of BCG vaccination in HIV-infected children3535 Fallo A, Torrado L, Sanches A, et al. Delayed complications of Bacille Calmette-Guerin (BCG) vaccination in HIV-infected children. In: 3 IAS Conference on HIV Pathogenesis and Treatment. 2005 jul 25-27. Rio de Janeiro: International AIDS Society; 2005.. The second was conducted at a South African hospital and concluded:

[…] the risk of disseminated BCG disease increases by several hundredfolds in HIV-infected infants, compared to the documented risk in HIV-uninfected infants3636 Hesseling AC, Marais BJ, Gie RP, et al. The risk of disseminated Bacille Calmette-Guerin (BCG) disease in HIV-infected children. Vaccine. 2007; 25(1):14-8.(14).

Revising the guidelines and changing the vaccine recommendation in these cases show the relevance of EBM practice to keep up with the advancement of medical knowledge. The findings of the two mentioned studies are the evidence that triggered the update of guidelines, which, in turn, led to the update of the online clinical text found in DynaMed Plus. The MS glossary is not guided by such practices, as there is no information on cases of contraindication for the BCG vaccine. Moreover, the documents that formed the basis for DynaMed Plus were published at least seven years ago.

This third case is that of the ‘not found’ indicator. The lack of information prevailed in our assessment in 29 out of 43 indicators used. Given this reality, we consider it relevant to make a few more comments on this case. There needs to be more information on all five dimensions defined for TB. Information on the BCG vaccine is missing under ‘prevention’.

Moreover, no data is available on prevention aimed at people living with HIV, children, and those responsible for children who are vulnerable to tuberculosis in some way. Regarding ‘transmission’, no information on risk factors for children and infection with resistant tuberculosis was found. Data on specific symptoms for the child population is also missing. Specific information for the HIV-infected population needs to be included under the ‘diagnosis’. This dimension needs more information on the child population and resistant tuberculosis. Finally, there is a lack of information on treatments aimed at the general population and people with HIV and information on the treatment of resistant tuberculosis and its side effects.

The fourth and final case is the indicator developed within the tuberculosis symptoms dimension and the ‘General’ group: ‘Symptoms suggestive of pulmonary tuberculosis are fever, fatigue, weight loss, night sweats, hemoptysis (blood in the sputum), or cough for two or more weeks; these symptoms combined with pleuritis (chest pain on inspiration and expiration) may also indicate tuberculosis’ (Id 133, table 2). We found the following information in the MS glossary:

What are the symptoms of tuberculosis?

  1. Cough for three weeks or more

  2. Evening fever

  3. Night sweats

  4. Slimming

The main symptom of pulmonary tuberculosis is coughing. This cough can be dry or productive (with phlegm).

IMPORTANT: We recommend that every person with respiratory symptoms, that is, with a cough for three weeks or more, be investigated for tuberculosis2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
.

We considered the glossary information to be ‘incorrect’ in this case. We returned to the DynaMed Plus website to verify the evidence supporting this indicator’s development. It is the result of the analysis of information from three topics: ‘Hemoptysis – Approach to the Patient’3737 DynaMed. Record No. T920582, Hemoptysis – Approach to the Patient. 2018. [acesso em 2019 jul 19]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T920582.
https://www.dynamed.com/topics/dmp~AN~T9...
, ‘Pleuritis – Approach to the Patient’3838 DynaMed. Record No. T922350, Pleuritis – Approach to the Patient. 2018. [acesso em 2019 jul 22]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T922350.
https://www.dynamed.com/topics/dmp~AN~T9...
, and ‘Pulmonary Tuberculosis’2323 DynaMed. Record No. T116300. Pulmonary Tuberculosis. 2018. [ acesso em 2019 jul 23]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T116300.
https://www.dynamed.com/topics/dmp~AN~T1...
. The evidence available on these topics makes three references.

The first is the book chapter presented in the first case of the indicator on TB transmission2828 Fitzgerald D, Sterling T, Haas D. Mycobacterium tuberculosis. In: Bennett J, Dolin RBM, editores. Mandell, Douglas, and Bennett’s Principles and Practice of Infectious Diseases. 8. ed. New York: Saunders; 2015. p. 2787-2818.. The second is a paper and concerns the evidence that allowed including pleuritis as one of the possible TB symptoms in the indicator1818 Zhang Y, Sun Y, Xie B. Quality of health information for consumers on the web: A systematic review of indicators, criteria, tools, and evaluation results. J Assoc Inf Sci Technol. 2015; 66(10):2071-84.. The wording of the indicator considers pleuritis as a symptom. However, this information is not included in the glossary. The third reference is the document entitled ‘International Standards for Tuberculosis Care, Edition 3’4040 World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc).
https://www.who.int/publications/m/item/...
, published in 2014 with funding from important institutions addressing TBs4141 Management Sciences for Health. Home. 2020 [acesso em 2020 jan 4]. Disponível em: https://www.msh.org.
https://www.msh.org...
. The document addresses diagnosis, treatment, and Public Health issues, especially norms. The point we highlight in this document is related to information about the cough symptom.

The discussion of the standard emphasizes the importance of including not only cough but also fever, night sweats, and weight loss as indicators for evaluating tuberculosis4040 World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc).
https://www.who.int/publications/m/item/...
(6).

There is a concern about highlighting other symptoms besides cough. In another part of the document, the following guidance appears in the description of the same standard:

All patients, including children, with an unexplained cough lasting two or more weeks or with suggestive unexplained TB findings on chest radiographs should be evaluated for tuberculosis4040 World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc).
https://www.who.int/publications/m/item/...
(9).

Then, the document justifies and summarizes the evidence supporting the standard. The data shown in the document consider that cough is one of the symptoms, but it is not the only or the main one.

According to the TB CARE I4040 World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc).
https://www.who.int/publications/m/item/...
, research indicates that 10 to 25% of patients with diagnosed tuberculosis report no cough. The document cites a study conducted in India with 55,561 patients that compared coughing for two or more weeks with coughing for three or more weeks as a reason for performing a diagnostic test4242 Santha T, Garg R, Subramani R, et al. Comparison of cough of 2 and 3 weeks to improve detection of smearpositive tuberculosis cases among out-patients in India. Int J Tuberc Lung Dis. 2005; 9(1):61-8.. It states that the investigation of patients who had a cough for two or more weeks increased the number of suspected TB cases by 61% and confirmed cases by 46%. Santha et al.4242 Santha T, Garg R, Subramani R, et al. Comparison of cough of 2 and 3 weeks to improve detection of smearpositive tuberculosis cases among out-patients in India. Int J Tuberc Lung Dis. 2005; 9(1):61-8. concluded that the screening criterion for sputum microscopy is cough after two weeks.

As mentioned, this fourth and last case is considered ‘incorrect’ according to the evaluation carried out on the page on TB in the MS glossary. Besides not presenting the hemoptysis and pleuritis symptoms, the glossary describes the cough symptom inaccurately. It characterizes cough as the primary symptom, while the evidence guides the importance of highlighting the other symptoms. Another issue is a persistent glossary with outdated information on the duration of cough as a symptom suggestive of TB. The study that identified the benefits of investigating tuberculosis in patients with a cough of two weeks or more dates from 20054242 Santha T, Garg R, Subramani R, et al. Comparison of cough of 2 and 3 weeks to improve detection of smearpositive tuberculosis cases among out-patients in India. Int J Tuberc Lung Dis. 2005; 9(1):61-8..

Cough is the traditionally known and reported symptom. For example, the official channel of the Ministry of Health on YouTube has a video of a campaign to encourage early TB diagnosis4343 Brasil. Ministério da Saúde. (472) #Tuberculose Filme oficial. [YouTube]. 2011. [acesso em 2019 ago 8]. Disponível em: https://www.youtube.com/watch?v=Dt5zJQsnWzw.
https://www.youtube.com/watch?v=Dt5zJQsn...
. It addresses only the cough as a symptom. As in the MS glossary, it is stated that coughing for three or more weeks may indicate TB, and the disease’s cure depends on early diagnosis. The duration of cough, available in the evidence used in the document4040 World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc).
https://www.who.int/publications/m/item/...
, is shorter than the time reported in the glossary2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
and the campaign4343 Brasil. Ministério da Saúde. (472) #Tuberculose Filme oficial. [YouTube]. 2011. [acesso em 2019 ago 8]. Disponível em: https://www.youtube.com/watch?v=Dt5zJQsnWzw.
https://www.youtube.com/watch?v=Dt5zJQsn...
. Thus, investigating TB in patients with cough from two weeks onwards could contribute to the early diagnosis of TB and the cure of the disease.

Final considerations

The four cases discussed confirm the importance of the indicators developed in this work based on the EBM practices1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.. The 43 information accuracy indicators can be translated into other languages and used to assess the quality of TB websites in any country.

In Brazil, the indicators can be used to update the MS TB page2626 Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose.
https://www.gov.br/saude/pt-br/assuntos/...
‘Health from A to Z’ glossary and align it with the best and most current scientific evidence. Evaluating the glossary with these indicators can be understood as a diagnosis of the quality of the information on this site. Those responsible for the website can access the result and verify the changes or inclusions of information that can be made.

In this sense, all topics in the ‘Health from A to Z’ glossary could be submitted to the same evaluation process. The methods can be replicated for other health issues contained in the glossary1919 Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.. They will facilitate retrieving reliable scientific evidence to develop information accuracy indicators on any health issue. This glossary evaluation work should be performed urgently to ensure that the website of this national reference public institution is constantly updated and provides correct information.

The problems of low quality and lack of information on portals such as the MS are directly related to the low importance of health information and communication on the Internet for public health managers and researchers. Brazilian science, technology, and innovation policymakers must be aware of new information and communication technologies’ role in societies around the planet.

In the fourth case discussed in this article, the investigation of patients who had a cough for two weeks increased the number of suspected TB cases by 61% and confirmed TB cases by 46%4040 World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc).
https://www.who.int/publications/m/item/...
. These data indicate a significant increase in early diagnosis and, consequently, in the early treatment of the disease. This essential information should be corrected in the MS ‘Health from A to Z’ glossary. Correct information per high information accuracy standards can prevent complications that lead to death and, thus, save lives.

Therefore, this paper addresses a relevant subject in the current academic setting, where the Internet has become one of the most soughtafter sources of health information, especially after the SARS-CoV-2 pandemic. However, interest in information security has remained the same. We recommend that further studies be carried out addressing this discussion.

  • Financial support: non-existent

Annex 1
First version of 180 ‘accuracy’ indicators for TB

References

  • 1
    Pereira Neto A, Barbosa L, Flynn M. Prefácio da edição brasileira – Há décadas em que nada acontece. Há semanas em que décadas acontecem. In: Pereira Neto A, Flynn M, organizadores. Internet e saúde no Brasil: desafios e tendências. 1. ed. São Paulo: Cultura Acadêmica; 2021. p. 7-18.
  • 2
    Pereira Neto A, Paolucci R. Avaliação da qualidade da informação de saúde na internet: análise das iniciativas brasileiras. In: Pereira Neto A, Flynn M, organizadores. Internet e saúde no Brasil: desafios e tendências. 1. ed. São Paulo: Cultura Acadêmica; 2021. p. 257-291.
  • 2
    Pereira Neto A, Paolucci R. Avaliação da qualidade da informação de saúde na internet: análise das iniciativas brasileiras. In: Pereira Neto A, Flynn M, organizadores. Internet e saúde no Brasil: desafios e tendências. 1. ed. São Paulo: Cultura Acadêmica; 2021. p. 257-291.
  • 4
    Cheng C, Dunn M. Health literacy and the Internet: a study on the readability of Australian online health information. Aust New Zeal J Public Heal. 2015; 39(4):309-14.
  • 5
    João BN, Souza CL, Serravo FA. A systematic review of smart cities and the internet of things as a research topic. Cad EBAPEBR. 2020; 17(4):1115-30.
  • 6
    Martinhão MS. Pesquisa sobre o uso das tecnologias de informação e comunicação nos domicílios brasileiros: TIC domicílios 2018. São Paulo: Comitê Gestor da Internet no Brasil; 2019. [acesso em 2022 out 5]. Disponível em: https://www.cgi.br/media/docs/publicacoes/2/12225320191028-tic_dom_2018_livro_eletronico.pdf
    » https://www.cgi.br/media/docs/publicacoes/2/12225320191028-tic_dom_2018_livro_eletronico.pdf
  • 7
    Barbosa AF. Pesquisa sobre o uso das Tecnologias de Informação e Comunicação nos domicílios brasileiros: TIC Domicílios 2020: edição COVID-19: metodologia adaptada. São Paulo: Comitê Gestor da Internet no Brasil; 2021. [acesso em 2022 out 5]. Disponível em: https://www.cgi.br/media/docs/publicacoes/2/20211124201233/tic_domicilios_2020_livro_eletronico.pdf
    » https://www.cgi.br/media/docs/publicacoes/2/20211124201233/tic_domicilios_2020_livro_eletronico.pdf
  • 8
    Park H, Lee S-I, Hwang H, et al. Can a health information exchange save healthcare costs? Evidence from a pilot program in South Korea. Int J Med Inf. 2015; 84(9):658-66.
  • 9
    Spoelman WA, Bonten TN, Waal MWM, et al. Effect of an evidence-based website on healthcare usage: an interrupted time-series study. BMJ Open. 2016; 6(11).
  • 10
    Eysenbach G. The impact of the Internet on cancer outcomes. CA-A CANCER J Clin. 2003; 53(6):356-71.
  • 11
    Garbin H, Guilam M, Pereira Neto A. Internet na promoção da saúde: um instrumento para o desenvolvimento de habilidades pessoais e sociais. Physis. 2012; 22(1):347-363.
  • 12
    Pereira Neto A, Ribeiro B, Guljor AP, et al. Eu quero entrar na rede: análise de uma experiência de inclusão digital com usuários do Caps. Saúde debate. 2020; (44):58-69.
  • 13
    Garbin HB, Pereira Neto A, Guilam MCR. A internet, o paciente expert e a prática médica: uma análise bibliográfica. Interface – Comum. Saúde, Educ. 2008; (26):579-88.
  • 14
    Lemos A. Cidade e mobilidade. Telefones celulares, funções pós-massivas e territórios informacionais. MATRIZes. 2007; 1(1):121-37.
  • 15
    Oliveira T. Como enfrentar a desinformação científica? Desafios sociais, políticos e jurídicos intensificados no contexto da pandemia. Liinc em Revista. 2020; (16):e5374.
  • 16
    Paolucci R, Neto AP. Methods for evaluating the quality of information on health websites: Systematic Review (2001-2014). Lat Am J Dev. 2021; 3(3):994-1056.
  • 17
    Eysenbach G, Powell J, Kuss O, et al. Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web. JAMA. 2002; 287(20):2691-700.
  • 18
    Zhang Y, Sun Y, Xie B. Quality of health information for consumers on the web: A systematic review of indicators, criteria, tools, and evaluation results. J Assoc Inf Sci Technol. 2015; 66(10):2071-84.
  • 19
    Paolucci R, Neto A, Nadanovsky P. Avaliação da acurácia da informação em sites de saúde: Métodos para construção de indicadores baseados em evidência. Em Questão. 2021; 27(4):137-88.
  • 20
    Brasil. Ministério da Saúde, Secretaria de Vigilância em Saúde. Boletim Epid. 2019 [acesso em 2018 jan 13]; (9). Disponível em: http://portalarquivos2.saude.gov.br/images/pdf/2019/marco/22/2019-009.pdf
    » http://portalarquivos2.saude.gov.br/images/pdf/2019/marco/22/2019-009.pdf
  • 21
    McMaster U. About. 2019. [acesso em 2019 maio 30]. Disponível em: https://www.accessss.org/Pages/About
    » https://www.accessss.org/Pages/About
  • 22
    DynaMed. About DynaMed Clinical Decision Support Tool. 2019. [acesso em 2019 maio 7]. Disponível em: https://www.dynamed.com/home/about
    » https://www.dynamed.com/home/about
  • 23
    DynaMed. Record No. T116300. Pulmonary Tuberculosis. 2018. [ acesso em 2019 jul 23]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T116300
    » https://www.dynamed.com/topics/dmp~AN~T116300
  • 24
    Paolucci R. Avaliação da qualidade da informação em sites de saúde: indicadores de acurácia baseada em evidência para tuberculose. [tese]. Rio de Janeiro: Fundação Oswaldo Cruz; 2020. 149 p.
  • 25
    Brasil. Ministério da Saúde. Saúde de A a Z – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-de-a-a-z
    » https://www.gov.br/saude/pt-br/assuntos/saude-de-a-a-z
  • 26
    Brasil. Ministério da Saúde. Tuberculose – Português (Brasil). 2022. [acesso em 2022 fev 24]. Disponível em: https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose
    » https://www.gov.br/saude/pt-br/assuntos/saude-dea-a-z/t/tuberculose/tuberculose
  • 27
    Pfyffer G, Palicova F. Mycobacterium: General characteristics, laboratory detection, and staining procedures. In: Versalovic J, Carroll K, Funke G, et al., editores. Manual of Clinical Microbiology. 10. ed. Washington: American Society for Microbiology Press; 2011. p. 472-502.
  • 28
    Fitzgerald D, Sterling T, Haas D. Mycobacterium tuberculosis. In: Bennett J, Dolin RBM, editores. Mandell, Douglas, and Bennett’s Principles and Practice of Infectious Diseases. 8. ed. New York: Saunders; 2015. p. 2787-2818.
  • 29
    Burwell DS. Acquiring the evidence: How to find current best evidence and have current best evidence find us. In: Straus SE, Glasziou P, Richardson WS, et al., editores. Evidence-based medicine: how to practice and teach EBM. 5. ed. Elsevier Health Sciences; 2019. p. 57-90.
  • 30
    DynaMed. Record No. T909352, Latent Tuberculosis Infection in Patients With HIV. 2019. [acesso em 2019 jul 22]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T909352
    » https://www.dynamed.com/topics/dmp~AN~T909352
  • 31
    Kwan C, Ernst JD. HIV and tuberculosis: A deadly human syndemic. Clin Microbiol Rev. 2011; 24(2):351- 76.
  • 32
    DynaMed. Record No. T905489, Bacille Calmette- Guerin (BCG) Vaccine. 2018. [acesso em 2019 jul 18]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T905489
    » https://www.dynamed.com/topics/dmp~AN~T905489
  • 33
    World Health Organization. Revised BCG vaccination guidelines for infants at risk for HIV infection. Wkly Epidemiol Rec. 2007; 82(21):193-6.
  • 34
    Stop TB. Partnership. The global plan to stop TB 2011- 2015: Transforming the fight towards elimination of tuberculosis. 2011 [acesso em 2022 out 5]. Disponível em: http://www.stoptb.org/assets/documents/global/plan/TB_GlobalPlanToStopTB2011-2015.pdf
    » http://www.stoptb.org/assets/documents/global/plan/TB_GlobalPlanToStopTB2011-2015.pdf
  • 35
    Fallo A, Torrado L, Sanches A, et al. Delayed complications of Bacille Calmette-Guerin (BCG) vaccination in HIV-infected children. In: 3 IAS Conference on HIV Pathogenesis and Treatment. 2005 jul 25-27. Rio de Janeiro: International AIDS Society; 2005.
  • 36
    Hesseling AC, Marais BJ, Gie RP, et al. The risk of disseminated Bacille Calmette-Guerin (BCG) disease in HIV-infected children. Vaccine. 2007; 25(1):14-8.
  • 37
    DynaMed. Record No. T920582, Hemoptysis – Approach to the Patient. 2018. [acesso em 2019 jul 19]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T920582
    » https://www.dynamed.com/topics/dmp~AN~T920582
  • 38
    DynaMed. Record No. T922350, Pleuritis – Approach to the Patient. 2018. [acesso em 2019 jul 22]. Disponível em: https://www.dynamed.com/topics/dmp~AN~T922350
    » https://www.dynamed.com/topics/dmp~AN~T922350
  • 39
    Reamy BV, Williams PM, Odom MR. Pleuritic Chest Pain: Sorting Through the Differential Diagnosis. Am Fam Physician. 2017; 96(5):306-12.
  • 40
    World Health Organization. International Standards for Tuberculosis Care. 3. ed. 2014. [acesso em 2019 dez 5]. Disponível em: https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc)
    » https://www.who.int/publications/m/item/international-standards-fortuberculosis-care-(istc)
  • 41
    Management Sciences for Health. Home. 2020 [acesso em 2020 jan 4]. Disponível em: https://www.msh.org
    » https://www.msh.org
  • 42
    Santha T, Garg R, Subramani R, et al. Comparison of cough of 2 and 3 weeks to improve detection of smearpositive tuberculosis cases among out-patients in India. Int J Tuberc Lung Dis. 2005; 9(1):61-8.
  • 43
    Brasil. Ministério da Saúde. (472) #Tuberculose Filme oficial. [YouTube]. 2011. [acesso em 2019 ago 8]. Disponível em: https://www.youtube.com/watch?v=Dt5zJQsnWzw
    » https://www.youtube.com/watch?v=Dt5zJQsnWzw

Publication Dates

  • Publication in this collection
    06 Jan 2023
  • Date of issue
    Oct-Dec 2022

History

  • Received
    15 July 2022
  • Accepted
    21 Sept 2022
Centro Brasileiro de Estudos de Saúde RJ - Brazil
E-mail: revista@saudeemdebate.org.br