Claudia Travassos

Debate sobre o artigo de Zulmira M. A. Hartz

Debate on the paper by Zulmira M. A. Hartz



Escola Nacional de Saúde Pública, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil.  



In this interesting article, Zulmira Hartz traces the history of the institutionalization of policy and program evaluation in France as a reference to reflect on possible paths to be followed in Brazil. The definition of institutionalization adopted by the author (integration of evaluation into the organizational system one is attempting to influence) expresses the evaluation approach she assumes, that is, an activity not limited to research practice, but an integral part of policy-making and implementation of activities in these policies and programs. In this approach, the role of evaluators and/or evaluation agencies is expanded by assuming the additional function of facilitating quality improvement processes. The author thus draws the concepts of evaluation and regulation closer together. Under a broad approach, regulation is seen not only as a normative resource, but also as an action facilitating governance (Almeida et al., 1998). According to this approach, the effectiveness of evaluation implies the use of less coercive (and thus more participatory) strategies, in addition to creating flexible, decentralized evaluation structures.

In the specific field of evaluating quality of health care services, an integral part of evaluating health programs and policies, one also observes, in many countries, relevant changes in quality evaluation relating to the changes identified by Hartz vis-à-vis program and policy evaluation. Such changes result from the realization that current health care production requires quite complex systems and processes, and thus that the results of patient care are largely explainable by problems in these systems and processes and not merely by the performance of a specific health care provider (physician or nurse). Since improved performance by health care organizations depends fundamentally on actions developed by the organization itself, regulatory and quality evaluation agencies should act to motivate organizations to improve their performance, leaving sanctions for those cases in which their is evident risk to the public. An example of this new approach was led in the early 1990s by the Joint Commission for Accreditation of Health Care Organizations (JCAHCO), a traditional American institution in this field. The JCAHCO redefined its evaluation focus, concentrating more on seeking health care services' conformity to standards targeting processes and activities with a direct or indirect effect on patient care, improving their communications with health services and emphasizing their educational role, in addition to transmitting evaluation results to the public.

Despite agreeing with the concept of evaluation adopted by the author, I would like to introduce an alternative to the emphasis placed on institutionalization in her article. In expanding the concept of evaluation, I wish to do so with caution in order not to underrate the importance of research for the effectiveness of the evaluation process. In the specific case of health care services, evaluating quality of care is an extremely difficult task, given certain characteristics of physician practice, such as the fact that it is based on specialized knowledge, permeated by uncertainties with regard to a major portion of the available diagnostic and therapeutic procedures, and the fact that patients present broad variability in the risks of developing adverse results. Research in health care services is a relatively new interdisciplinary field still dealing with important technical and methodological limitations, despite advance achieved in the last two decades (Brook, 1996). In addition, in this new paradigm, the way the results are summarized and publicized gains relevance, which implies constituting evaluation teams with new competencies, such as communications, teaching, and policy-making.

Thus, from my point of view, institutionalization of evaluation in the health sector should focus not only on improving interaction between decision-makers, evaluators, and health care managers and providers. It should also consider strategies and resources needed to encourage the production of knowledge and training specialized personnel. Research producing imprecise results leads to conflicting interpretations and fails to produce discernment of the facts analyzed. It can thus generate discredit over the usefulness of evaluation activities, with a negative impact on the value ascribed to them by decision-makers, health care managers and providers, and users.

Although in the French case the process of institutionalizing evaluation began late and was less prominent than in other countries cited by the author (due to the French political and academic culture and not a lack of specialists), I believe this was not so for Brazil. Our Brazilian reality is more complicated: we lack a political culture oriented towards evaluation, and I think that despite recent efforts, we still experience a chronic and severe lack of academic and technical specialists in quality evaluation, quality management, production, and analysis of health data, medical documentation, and so forth. Thus, as pointed out by the author in quoting Gérard de Pourvourville, we should search for shortcuts, learning from the experience of other countries, but shaping it to our mesures, which unfortunately still express a multiplicity of deficiencies.

ALMEIDA, C.; TRAVASSOS, C.; PORTO, S. & BAPTISTA, T., 1988. A Reforma Sanitária Brasileira: em Busca da Eqüidade. Relatório final de pesquisa, Rio de Janeiro: Escola Nacional de Saúde Pública, Fundação Oswaldo Cruz.

BROOK, R.; McGLYN, E. & CLEARY, P., 1996. Quality of care. Part 2: Measuring quality of care. New England Journal of Medicine, 13:966-970.

Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz Rio de Janeiro - RJ - Brazil