Abstract

Working relations between the Pan- American Health Organization/World Health Organization (PAHO/WHO) and Brazilian health institutions accumulated a long history of cooperation with mutual benefits, which in many cases were shared with other nations under various cooperation frameworks among countries for health development. A milestone in this relationship is the technical cooperation provided by PAHO/WHO to the More Doctors Program (Programa Mais Médicos - PMM). This cooperation has added both strategic value in reducing gaps in health equality and has capitalized on the unique nature of the Cuba-Brazil South-South cooperation experience, triangulated through PAHO/WHO. This paper discusses PAHO/WHO's role in the evaluation of its technical cooperation within PMM. A Monitoring and Evaluation (M&E) Framework has been developed in order to progressively identify the advances in coverage and quality of primary health care provided by the Unified Health System (Sistema Único de Saúde - SUS) through the PMM. Special attention was given to identify best practices in health services, to analyze results and impacts of the PMM, and to manage and share knowledge that has been produced by its implementation, through a web-based knowledge platform. Some relevant results of PMM are briefly presented and discussed.

Key words
More Doctors; Primary Health Care; Health Policy Evaluation

Introduction

“What is important is to never stop questioning.” This quotation, attributed to Albert Einstein, summarizes the meaning of the process being developed by the Pan-American Health Organization/World Health Organization (PAHO/WHO) to monitor and evaluate the More Doctors Cooperation Program (Programa Mais Médicos – PMM). It is necessary to pose pertinent questions and obtain answers that help the PAHO/WHO make the most adequate decisions for the Project. At the same time, it is indispensable to generate valuable knowledge in order to perfect the Unified Health System (SUS) and improve the life of the population served, as well as establish practical criteria for the contracting and international mobilization of professional doctors.

In the same way, keeping in mind the nature of an international organization in health cooperation, the PAHO/WHO hopes that the lessons learned as a result of this evaluation are useful for other countries in the Americas, dedicated to the broadening and strengthening of Primary Healthcare¹1. Organización Panamericana de la Salud (OPS). Renovación de la Atención Primaria de Salud en las Américas: documento de posición de la Organización Panamericana de la Salud. Washington: OPS; 2007..

Among the inquiries raised when we decided to carry out this research, four were selected with the aim of defining our ideas and identifying the essential means for executing this task: “what,” “when”, “where,” and “how.” Our proposal was to delimit, in the best possible way, the goal of evaluation, and define the timeframe, spaces, and strategies of its components.

The Cooperative Project of the PAHO/WHO with the PMM is complex, as it implies a great mobilization of human and financial resources subject to a permanent negotiation and coordination among the involved parties – the nation, states, municipalities, community organizations, and international cooperatives – with the aim of achieving success for this large-scale health intervention, intended to strengthen the Family Health Strategy and the SUS.

In the present article, we briefly present the central ideas and reflections that guide the development of the process, in addition to the findings obtained up to the present moment. In the same way, we list the future expectations and also our doubts and concerns about the subject, some of which have not been clarified because they require more time in the life of the PMM²2. Brasil. Lei Nº 12.871, de 22 de outubro de 2013. Institui o Programa Mais Médicos, altera as Leis no 8.745, de 9 de dezembro de 1993, e no 6.932, de 7 de julho de 1981, e dá outras providências. Diário Oficial da União 2013; 23 out. and PCMM³3. Organização Pan-Americana da Saúde (OPAS), Organização Mundial da Saúde (OMS). TC 80: Cooperação Técnica entre o Ministério da Saúde e a Organização Pan-Americana da Saúde - Projeto Acesso da população Brasileira à Atenção Básica em Saúde. Brasília: OPAS; 2013. programs.

Our first question was about what should be the object of monitoring and evaluation. At the same time, we recognize the importance to establishing limits to this process. This does not mean evaluating the national strategy, but rather to point out its advances and understand the findings of the Project, always keeping in mind that such an exercise would generate challenges as well as opportunities to improve the Program.

The boundaries between the PMM and the PCMM are, in certain respects, imprecise. The main goal of the Project is to ensure the presence of doctors in the health teams of Primary Care by way of the mobilization of Cuban physicians to Brazil and, simultaneously, commit to the successful performance of these health service professionals, thus generating an indissoluble link between the Program and Cooperative Project.

To better illustrate the complexity of relations between the parts involved, and the boundaries and convergence of responsibility between both, we use the classic categories proposed by Avedis Donabedian⁴4. Donabedian A. Evaluating the quality of medical care. The Milbank Memorial Fund Quarterly 2005; 83(4):691-729. for the evaluation of quality in health services: structure, process, and results (Chart 1). We define measurement indicators for each of them, which allows us not only to better understand the areas of overlap of the Program and the Project, but also to consider the existing articulations and synergies between them.

The evaluation of the structure of the PCMM is nearly entirely the responsibility of the PAHO/WHO. The aim of measurement concerns Cuban doctors: who they are, how many, where, and how they are placed and work. The execution of the Project's funding, which allows for the arrival and movement of the doctors, is equally the responsibility of the PAHO/WHO. On the other hand, it is worth recognizing the existence of the responsibilities shared between the PAHO/WHO and the Ministries of Health of Brazil and Cuba, such as decisions taken in a tripartite form after revision of the coordinated administration among the three parts regarding the location and movement of the doctors.

Regarding the processes, the convergence of interests shared between the PCMM and the PMM is notable. It is worth noting the quantification of services rendered, the training and evaluation of member doctors (Cuban doctors mobilized by way of PAHO/WHO cooperation), the work with the primary care teams, and the coordination with other units and services of the care network, cases in which the responsibilities are shared, in that the doctors work in the SUS environment.

Something similar occurs with the measurement of results. The performance of the professionals, their approval on the part of the medical community, the quality of their work, the contribution for the implementation of the family health care model, and the health results attributable to their work are in the common interest of the PMM and the PCMM, and are necessarily evaluated by all of the partners.

The monitoring and evaluation of the PCMM are processes unrelated to the execution of the PMM. The results of the evaluation unite and draw the interest of both projects in the development of analyses, the characterization of performance, and the results attributable to this intervention.

The second question dealt with the deadlines relative to the process of monitoring and evaluation. The strategic administration demands that, within the pertinent period, we obtain reliable information about the advances and the successes achieved, as well as the difficulties confronted, and that it is particularly important to evaluate the health impact of the PMM interventions. Frequently, these elements end up provoking tensions between the political and technical factions. In order to transcend these difficulties, the findings obtained in the short and medium term are thoroughly documented, such as the improvement of access and coverage, equity, resolubility, continuity and wholeness of care, and satisfaction of users. More time will be necessary for us to be able to evaluate the impact on indicators of mortality, morbidity, and quality of care, as well the sustainability of the project (Figure 1).

The selection of territorial and population areas with a view toward monitoring and evaluation constitutes a third guideline for this process. The PCMM is present in the 27 federative units of the country, in close to 4,000 municipalities, and in 34 indigenous districts. Brazil is a large and diverse country, and because of this the PMM develops in distinct realities that condition its activity. The collaborating doctors work in large and populous municipalities like São Paulo, and also in small municipalities with less than 5,000 inhabitants. They work in urban and rural zones, as well as in indigenous, quilombola (descendants of escaped slaves, similar to maroons), and urban-peripheral communities, which are culturally different from each other.

In some municipalities, the collaborators represent 100% of the doctors that work in basic care, while in other localities they equal only 5% or less. In certain health units, they work together with doctors from Brazil and other nationalities, but in other installations they constitute the totality of doctors present.

These differences are important when it comes time to proceed with the comparative analyses of findings among different territories. The solution was to stratify the universe of PCMM activity and combine quantitative and qualitative methodologies during the monitoring and evaluation. Criteria were defined for measuring the changes and differences between regions and municipalities, complementing this analysis with case studies, which allows us to deepen the experiences and better document those advances that are difficult to show by way of quantitative expressions.

Finally, it was necessary to respond to the broader question of how to achieve an efficient process of monitoring and evaluation of the PCMM, considering the specific context in which the Project was begun and had been developing in the country, where stances of a political nature against certain governmental institutions and corporatist criteria punctuate a complex behavior by the media, with actors both in favor and against the Program and, by extension, the Project.

The technical decision adopted was to develop a strategy that takes into account multiple sources of information and knowledge about the advance and the findings of the PMM, in addition to the studies carried out during the execution of the Project. To this end, the PAHO/WHO formulated the “Chart for Monitoring and Evaluation of the PAHO/WHO Cooperative Project with the Mais Médicos Program⁵5. Organização Pan-Americana da Saúde (OPAS), Organização Mundial da Saúde (OMS). Marco para o monitoramento e avaliação do Projeto de Cooperação Mais Médicos da OPAS/OMS. Brasília: OPAS; 2015.”, which considers three “macro areas” of work (Figure 2).

The first macro area is the direct responsibility of the Organization, centered on monitoring the process of insertion, distribution, retention, and training of collaborating doctors. The second deals with the performance and work conditions of the professionals in basic healthcare services and the health outcomes associated with their performance. The third has as its objective the evaluation of the impact and sustainability of the PCMM. To reach the goals of macro areas 2 and 3, PAHO/WHO uses knowledge generated by studies carried out by academic and research institutions (principally Brazilian ones), and establishes cooperation agreements with some of them and with the SUS administrators responsible for the PMM.

Through the initiative of the PAHO/WHO Regional Office, an assistance group was created, comprised of international specialists that accompany the process of monitoring and evaluation, with the capability of presenting recommendations geared to the betterment of the mentioned procedure.

After almost three years of the PMM, it is possible to determine important advances in the achievement of objectives and targets established by the program.

We construct strong partnerships with academic and research institutions in the country, with administrators at different levels within SUS and with the Brazilian Association of Collective Health (Associação Brasileira de Saúde Coletiva - Abrasco), in their role of a representative association of the Brazilian healthcare movement. Together with these institutions, we work in various directions, carrying out studies that use quantitative and qualitative methodologies, which allows us to measure advances, as well as identify and promote good practices and success stories.

A partnership with the “Abrasco Network for Research in Primary Healthcare”⁶6. Rede de Pesquisa APS [Internet]. Rede de Pesquisas em Atenção Primaria à Saúde. [acessado 2016 mar 25] Disponível em: http://www.rededepesquisaaps.org.br
http://www.rededepesquisaaps.org.br... was established, with the goal of promoting the realization of work that evaluates different PMM experiences, and which would be in the interest of the cooperative project. A seminar was held, bringing together many of the researchers of the Network that work in this area.

Another line of action is the creation of a web-based knowledge platform of the PMM, in partnership with the Information Center in Health Sciences of Latin America and the Caribbean (Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde - Bireme), with collaboration from the Ministry of Health, National Council of Health Secretariats (Conselho Nacional dos Secretários de Saúde - CONASS), National Council of Municipal Health Secretariats (Conselho Nacional de Secretarias Municipais de Saúde - CONASEMS), and Abrasco, taking advantage of the partnership with the Abrasco Network for Research in Primary Healthcare. This web-based platform is accessible by any interested user⁷7. Rede de Pesquisa APS [Internet]. Plataforma de conhecimentos do Programa Mais Médicos. [acessado 2016 mar 25]. Disponível em: http://apsredes.org/mais-medicos/
http://apsredes.org/mais-medicos/... . At the moment, it contains more than 150 texts, between reports of research in progress and scientific publications.

Simultaneously, the PAHO/WHO supports case studies that systematize success-story experiences of the PMM, with the purpose of improving knowledge andsharing the experiences and lessons learned, nationally and internationally. The experience of Curitiba⁸8. Organização Pan-Americana da Saúde (OPAS), Organização Mundial da Saúde (OMS). Implementação do Programa Mais Médicos em Curitiba. Experiências inovadoras e lições aprendidas. Brasília: OPAS; 2015. was finalized and published, and another four case studies are in preparation in partnership with the universities, the Ministry of Health, and the Secretariats of Health of the states of Rio de Janeiro⁹9. Organização Pan-Americana da Saúde (OPAS). Programa Mais Médicos para o Brasil no Município do Rio de Janeiro: Mais Acesso, Equidade e Resolutividade na APS. Brasília: OPAS; 2016., Minas Gerais¹⁰10. Organização Pan-Americana da Saúde (OPAS). A Atenção Básica à Saúde e o Programa Mais Médicos em Minas Gerais: um estudo de casos múltiplos. Brasília: OPAS; 2016., Rio Grande do Norte¹¹11. Organização Pan-Americana da Saúde (OPAS). Atenção à saúde em municípios de pequeno porte do Rio Grande do Norte: efeitos do programa Mais Médicos para o Brasil. Brasília: OPAS; 2016., and Maranhão¹²12. Organização Pan-Americana da Saúde (OPAS). Atenção à saúde em municípios de pequeno porte do Maranhão: efeitos do Programa Mais Médicos para o Brasil. Brasília: OPAS; 2016..

In considering the outcomes achieved by the PMM up to now, we cannot forget that it is based on robust evidence of management produced by an analysis of the impact of the Family Health Strategy. Studies clearly show that the increase in coverage of the population with family health teams has a positive effect on health indicators, thus contributing to a reduction in the rate of infant mortality and in hospitalizations on account of outpatient care¹³13. Macinko J, Guanais F, Marinho MF. Evaluation of the impact of the Family Health Program on infant mortality in Brazil, 1990–2002. J Epidemiol Community Health 2006; 60(1):13-19.,¹⁴14. Macinko J, Harris MJ. Brazil's Family Health Strategy. Delivering Community-Based Primary Care in a Universal Health System. N Engl J Med 2015; 372(23):2177-2181.. According to specialists in public health program evaluations, in the case of interventions of proven effectiveness it would be sufficient to demonstrate that the intervention was adequately conducted and reached the target public¹⁵15. Habicht JP, Victora CG, Vaughan JP. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int. J Epidemiol 1999; 28(1):10-18.. We consider that this assumption is valid for the case of the PMM, intended to strengthen the basic healthcare teams and the family healthcare model, whose effectivity has been proven by studies and recognized by academic entities and international health organizations.

Nearly three years since its start, the impact of the PMM is evident in the lives of millions of Brazilians. For the first time, inhabitants of more than 700 small Brazilian cities can rely on a resident doctor in their territory and do not need to travel to another municipality in search of medical attention or to pay for specialized services.

Until December of 2015, 12,446 doctors were recruited (including terminations and replacements). In the same period, 11,404 professional doctors were active, meeting the demands of the Ministry of Health. By December of 2015, of the total 12,446 doctors provided by the project, 880 (7.07%) collaborating doctors were dismissed from the Project for different motives related to the noncompliance of their conditions, responsibilities, and duties¹⁶16. Organização Pan-Americana da Saúde (OPAS). Segundo Informe Técnico Monitoramento do TC 80 Projeto Mais Médicos Ano 2015. Brasília: OPAS; 2015. [Documento interno].

One of the goals of the PMM is the betterment of doctors in primary healthcare via the offering of specialization courses by public institutions of higher education and through teaching, research, and extension activities. Thus, when the doctors are inserted into the Program, they are linked to a national institution of higher learning to receive training content and improvement. In December of 2015, 10,115 (88.7%) of the 11,404 total doctors had completed or were taking specialization courses¹⁶16. Organização Pan-Americana da Saúde (OPAS). Segundo Informe Técnico Monitoramento do TC 80 Projeto Mais Médicos Ano 2015. Brasília: OPAS; 2015. [Documento interno].

Graph 1 shows the distribution of doctors according to the main regions of Brazil, privileging the North region, followed by the Northeast and South, which should correspond to a more equitable access to basic healthcare.

The priority regions for the allocation of doctors were classified as areas difficult to access, difficult for the provisioning of doctors, or that possess greatly vulnerable populations. Table 1, with data up until December of 2015, shows that 84.85% of doctors were allocated in priority areas by SUS.

In the same way, the doctors of the Program are present in the 34 indigenous districts of the country, caring for an extremely vulnerable population that is traditionally forgotten by social services (Graphic 2). These are hugely important conquests for the materialization of the right to health as established in the Brazilian Constitution of 1988¹⁷17. Brasil. Constituição da República Federativa do Brasil de 1988. Diário Oficial da União 1988; 5 out..

The impact of the PMM on SUS is equally evident. Between August of 2013 and January of 2016, the estimated coverage of the population cared for by family health teams grew from 55.75% to 63.85%, which represents an increase of 14.4%¹⁸18. Brasil. Ministério da Saúde (MS). Departamento de Atenção Básica [Internet]. Histórico de Cobertura da Saúde da Família. [acessado 2016 mar 29]. Disponível em: http://dab.saude.gov.br/portaldab/historico_cobertura_sf.php
http://dab.saude.gov.br/portaldab/histor... . According to the report of the Federal Court of Auditors of 2014, which studied a sample of municipalities comparing two periods (before and after the arrival of the PMM), there was a growth of 33% in the monthly average of doctor visits in the municipalities benefiting from the PMM, a higher value than the 14% increase in the municipalities that did not receive doctors from the Program. The same report showed an increase of 32% in the house calls attributed to the PMM in one group of Basic Health Units (Unidades Básicas de Saúde – UBS) visited¹⁹19. Tribunal de Contas da União (TCU). Auditoria operacional: Programa Mais Médicos e Projeto Mais Médicos para o Brasil; avaliação da eficácia do programa. TC nº 005.391/2014-8. [Internet] Brasília, DF: Tribunal de Contas da União, 2015. [acessado 2016 mar 24]. Disponível em: http://www.tcu.gov.br/Consultas/Juris/Docs/judoc/Acord/20150702/AC_1605_25_15_P.doc
http://www.tcu.gov.br/Consultas/Juris/Do... . We can also note a greater resolubility of the services where the Program doctors work, due to the broader range of procedures offered at the first level of care. More relevant than the increase in the production of services, which was to be expected with the increase in their provision, the PMM has a positive impact in the implementation of the care model proposed by the Family Health Strategy for the SUS. Partial results of a study carried out by a team of researchers in poor municipalities of five Brazilian regions reveal positive effects of the Program in this direction²⁰20. Santos L, Costa AM, Girardi S. Programa Mais Médicos: uma ação efetiva para reduzir iniquidades em saúde. Cien Saude Colet 2015; 20(11):3547-3552.. These positive results regarding the family healthcare model coincide with those of the case study carried out by PAHO/WHO in Curitiba cited earlier.

The satisfaction and acceptance of the users in relation to foreign professionals constitutes another area with very relevant results for the evaluation of the PMM, due to the doubts previously raised about this question. It was said that the population would have problems with the doctors due to factors linked to differences of language, customs, and modus operandi. Today, we have evidence that allows us to overturn this myth.

Results from research carried by the Federal University of Minas Gerais, which spoke with14,000 people in 700 municipalities, show that 94% of those interviewed are satisfied or very satisfied with the PMM²¹21. Brasil. Programa tem aprovação de 94% das pessoas atendidas. Portal Brasil. [Internet] 2015. [acessado 2016 maio 20]. Disponível em: http://www.brasil.gov.br/saude/2015/08/populacao-medicos-e-gestores-aprovam-o-mais-medicos
http://www.brasil.gov.br/saude/2015/08/p... . These results coincide with those of other research and case studies in progress²²22. Associação Brasileira de Saúde Coletiva (Abrasco). Avaliação de Política Pública: o Programa Mais Médicos. [Internet] Plataforma de Conhecimentos Programa Mais Médicos. [acessado 2016 maio 20]. Disponível em: http://pesquisa.bvsalud.org/pmm/resource/pt/pesqpmm-95#
http://pesquisa.bvsalud.org/pmm/resource... .

These are only some examples of the enormous potential, in a vast range of outcomes, which a program like the PMM can produce. For a more systematic and profound analysis of the impact of the PMM, the PAHO/WHO implemented the “Framework for Monitoring and Evaluation of the PAHO/WHO Cooperative Project with the Mais Médicos Program,” cited above, which proposes a broad measurement, capable of capturing the complexity of the intervention in the medium and long term.

Final considerations

The PCMM is an innovative initiative of cooperation of the PAHO/WHO, bearing in mind the area of interest involved – international mobilization of professionals for a public health system – and the dimensions that such contracting achieved: 11,400 doctors in a period of three years. To develop this process of cooperation demanded a continuous political and technical administration between the Organization and the two participating countries, Brazil and Cuba, carried out through their Health Ministries with the involvement of other government entities, such as the ministries of Foreign Relations, as well as state and municipal authorities.

The Project is part of the cooperation between countries for the development of health, driven by the PAHO/WHO,²³23. Organização Pan-Americana da Saúde (OPAS). Cooperação entre Países para o Desenvolvimento da Saúde. [Internet] [acessado 2016 maio 19]. Disponível em: http://www.paho.org/bra/index.php?option=com_content&view=category&id=1041&layout=blog&Itemid=608
http://www.paho.org/bra/index.php?option... and has antecedents in other cooperation projects that involve mobilization of health professionals from Cuba in triangulation with PAHO/WHO. We draw attention to the support to African countries to face the Ebola epidemic²⁴24. Organización Mundial de la Salud (OMS). La OMS agradece apoyo de médicos cubanos en la respuesta al Ebola en África Occidental. [Internet] Centro de prensa de la OMS. Declaración de la OMS. Genebra, 2014. [acessado 2016 maio 17]. Disponível em: http://www.who.int/mediacentre/news/statements/2014/cuban-ebola-doctors/es/
http://www.who.int/mediacentre/news/stat... , the collaboration with Angola for the eradication of polio²⁵25. Marimón N, Martinez Cruz E. La cooperación técnica OPS/OMS y el aporte de Cuba en la erradicación de la polio en Angola. [Internet]. Rev Cubana Hig Epidemiol 2009 jan-abr [acessado em 2016 maio 17];47(1). Disponível em: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1561-30032009000100008
http://scielo.sld.cu/scielo.php?script=s... , and the mobilization of health personnel in the case of emergencies for disasters, such as the recent episode of the 2010 earthquake in Haiti²⁶26. Pan-American Health Organization (PAHO). Emergency Medical Teams. [Internet] Emergency Preparedness and Disaster Relief. [acessado 2016 maio 19]. Disponível em: http://www.paho.org/disasters/index.php?option=com_content&view=category&layout=blog&id=1216&Itemid=1136&lang=en.
http://www.paho.org/disasters/index.php?... .

However, the PCMM is a unique experience for the Organization by means of managing the financial resources of one Member-State, Brazil, to mobilize health professionals of Cuba who fully integrated into the SUS of the country. The significant mobilization of Cuban doctors and the complexity of the process of recruitment, preparation, and operation coordination between the two countries and the PAHO/WHO, marks this project with a singular character for the Organization.

To become a candidate for a job in the Program, the Cuban doctors must be specialists in Comprehensive General Medicine, with ten years of professional experience, a minimum of two years of work in another country, and basic understanding of Portuguese. While in the Program, the doctors undertake a host modulethat deals with themes regarding the functioning of SUS, protocols of primary care in Brazil, and the Portuguese language, with a duration of three weeks and minimum workload of 120 hours. In initiating their professional activities, the doctors participate in a specialization course of family and community health, guided by tutors and supervisors.

The monitoring and evaluation of the PCMM offers significant opportunities to the PAHO/WHO, since at the same time that it serves as an appropriate instrument to accompany the management of the project and report on the activity of the Organization, it also constitutes a mechanism that generates knowledge that will contribute to the operational improvement of SUS and its services to the population, mainly at the level of primary care, as well as for better discernment in the determination of the priorities of policies and investments towards the development of health in the country.

The PCMM will leave an innovative balance of knowledge and practices regarding a complex theme in the realm of international health: the international contracting of health professionals. The traditional flows of migration of medical professionals normally occurs from countries of lesser development to countries of greater economic development and better professional remuneration. The PMM shows the existence of another direction, in proposing a temporary migration (for periods of three years) of professionals originating in countries where the ratio of doctors to inhabitants is superior than that in Brazil. Aside from this, the program offers a specialized academic training in Family and Community Medicine, and the experience of working in a consolidated and standardized public health system, as well as learning in the Portuguese language.

The cooperation of the PAHO/WHO by way of the PCMM is not limited only to temporarily furnishing Cuban doctors to the country, although this has been the most visible component in the first two years of its activity. There are other modalities of cooperation in progress, such as technical assistance directed to the different sectors of SUS, the facilitation of cooperation agreements between countries for the development of health, the administration of knowledge concomitantly with carrying out a process of monitoring and evaluation, educational activities for the doctors, and the actions of social communication of the Program and the Cooperation Project.

References

Publication Dates

History