SciELO - Scientific Electronic Library Online

 
vol.79 issue11The contribution of reproductive ill-health to the overall burden of perceived illness among women in southern IndiaThe burden of disease and injury in Australia author indexsubject indexarticles search
Home Page  

Bulletin of the World Health Organization

Print version ISSN 0042-9686

Bull World Health Organ vol.79 n.11 Genebra Jan. 2001

http://dx.doi.org/10.1590/S0042-96862001001100012 

Policy and Practice


Syndromic treatment of gonococcal and chlamydial infections in women seeking primary care for the genital discharge syndrome: decision-making

Frieda M-T.F. Behets,1 William C. Miller,1 & Myron S. Cohen1

 

 


Abstract The syndromic treatment of gonococcal and chlamydial infections in women seeking primary care in clinics where resources are scarce, as recommended by WHO and implemented in many developing countries, necessitates a balance to be struck between overtreatment and undertreatment. The present paper identifies factors that are relevant to the selection of specific strategies for syndromic treatment in the above circumstances. Among them are the general aspects of decision-making and caveats concerning the rational decision-making approach. The positive and negative implications are outlined of providing or withholding treatment following a specific algorithm with a given accuracy to detect infection, i.e. sensitivity, specificity and predictive values. Other decision-making considerations that are identified are related to implementation and include the stability of risk factors with regard to time, space and the implementer, acceptability by stakeholders, and environmental constraints. There is a need to consider empirically developed treatment algorithms as a basis for policy discourse, to be evaluated together with the evidence, alternatives and arguments by the stakeholders.

Keywords Vaginal discharge/therapy; Cervix diseases/therapy; Neisseria gonorrhoeae; Chlamydia trachomatis; Syndrome; Risk factors; Decision support techniques; Algorithms (source: MeSH).

Mots clés Perte vaginale/thérapeutique; Col utérin, Maladies/thérapeutique; Neisseria gonorrhoeae; Chlamydia trachomatis; Syndrome; Facteur risque; Système aide décision; Algorithme (source: INSERM).

Palabras clave Excreción vaginal/terapia; Enfermedades del cuello uterino/terapia; Neisseria gonorrhoeae; Chlamydia trachomatis; Síndrome; Factores de riesgo; Técnicas de apoyo para la decisión; Algoritmos (fuente: BIREME).


 

 

Introduction

WHO recommends that patients who present for primary care with symptoms that suggest sexually transmitted infections (STIs) in settings where resources are scarce be treated syndromically using algorithms or clinical flowcharts (1). This approach was supported by the results of an intervention trial conducted in Mwanza, United Republic of Tanzania, which demonstrated a 38% decrease in the incidence of human immunodeficiency virus (HIV) infection associated with a syndromic STI primary care programme over a two-year period (2). In many countries, syndromic STI care is now a principal national HIV/STI prevention and control strategy.

The objectives of syndromic STI management are to provide rapid relief of symptoms, to treat all infections effectively, to avoid harm and unnecessary treatment, and to prevent future STIs in individual patients and communities. Although seemingly straightforward, these objectives may be mutually conflicting and may pose dilemmas for decision-makers.

Not all STI-related syndromes are equally amenable to syndromic management strategies. For example, genital discharge syndrome in women is particularly difficult to treat through the use of algorithms. It is difficult to differentiate the most commonly associated conditions of bacterial vaginosis, candidiasis, and trichomoniasis from the less frequent but more perilous cervical infections caused by Neisseria gonorrhoeae or Chlamydia trachomatis (3).

With a view to improving the syndromic management of the vaginal discharge syndrome, WHO has promoted the use of individual risk assessment (1). However, empirical studies on local risk factors for cervical infection have demonstrated the difficulties associated with such management of this syndrome. The use of risk factors can increase the sensitivity of an algorithm for the detection of cervical infection, but this is achieved at the cost of low specificity (4–6). Policy-makers thus have to make difficult decisions during the development of treatment guidelines.

The decision to select a particular treatment strategy for cervical infection is often based on explicit and implicit criteria. For example, one may decide to choose a highly sensitive approach in order to treat as many infections as possible, and to accept low specificity, which means accepting a substantial proportion of unnecessary treatment. Although presenting the decision criteria in a succinct manner without a thorough discussion of the implications has advantages, there are also substantial risks associated with simplifying complex matters. Simplification can make information more accessible and easier to communicate and it may lead to more uniform decisions. However, the discourse is narrowed and there is a risk of poorly informed decision-making. The selection of decision models involves a wide range of criteria and considerations that deserve careful scrutiny.

In this paper we do not offer answers to the dilemmas posed by syndromic care; instead we review factors that are relevant to the selection of particular strategies for the treatment of cervical infections caused by N. gonorrhoeae or C. trachomatis. Some general theoretical aspects of decision-making are outlined and we discuss caveats regarding the rational decision-making approach and the instruments of rationality that are generally used. Finally, we review specific selection criteria and vital considerations related to the delivery of health care.

 

Decision-making

Research conducted in order to identify risk factors or risk markers for cervical infection that may be useful in connection with clinical policy is part of a decision-making process. Operations research has been promoted by Simon as a means of gaining knowledge that would lead to more rational decision- making (7). Rationality provides a practical tool for policy analysis and the decision-making process (8): it implies structure, is widely used, allows comparisons and can lead to new insights. Clinical algorithms are themselves embodiments of rational decision-making. Instruments of rationality generally used in this type of operations research include epidemiology, statistics, and implicit notions of economics.

Among the limitations of the rational model acknowledged by Simon are the following: incomplete and fragmented knowledge, human constraints and behaviours, and restrictions of organizational environments. He therefore developed the bounded rationality concept, integrating reason and psychology or "affect" (9). Although the information gathered by scientific inquiry may be helpful in the decision-making process, informed choices are made on necessarily imperfect information, i.e. bounded rationality, influenced by the values and assumptions of the researcher, who decides what is "best" in accordance with specified summary criteria. While inquiry that relies on the analysis of quantitative data may appear objective, the data and their analysis, interpretation and use are not value-free and do not lead to a single right answer (10).

The empirical realities of decision-making in a complex and uncertain world are often puzzling and paradoxical (10). The variables that influence decision-makers as they manage conflicts include information, personal preferences, knowledge, skills, power, perceptions of issues, perceptions of options, consequences of choices, likelihoods of events and prevailing decision rules. In addition, decision- makers are surrounded by layers of influence and pressure composed of individuals, groups, organizations and agencies (8, 11).

 

Criteria for selecting algorithm

The criteria for selecting a particular algorithm for the case management of vaginal discharge syndrome should maximize the pursued goals at the individual and community levels and minimize costs to the individual and society. They should also include characteristics that optimize implementation. The selection of an algorithm for cervical infection is a lesson in trade-off because of the modest accuracy offered by empirically developed clinical decision models.

Ideally, and in line with the rational approach, a cost–benefit analysis should direct the selection of a particular algorithm. In practice, however, a meaningful cost–benefit analysis requires data that are generally unavailable or may not be measurable, e.g. local probabilities of STI transmission, developing sequelae and complications. It also requires highly debatable valuations of negative and positive health outcomes, e.g. values of infertility, mistaken suspicion that one's only sex partner is unfaithful). While an explicit cost–benefit analysis may not be feasible, it is possible to identify the factors that should influence the decision-making process and to consider them at least qualitatively.

 

Accuracy of algorithm

The accuracy or performance of a particular approach, based on sensitivity, specificity and predictive values, can be interpreted in the context of positive and negative implications of false- and true-positives and false- and true-negatives. Such positive and negative implications delineate the relevance of the sensitivities, specificities and predictive values of the decision models for the detection and management of cervical infection. An outline is given below of the positive and negative implications of providing or withholding treatment to patients according to a particular algorithm and the true infection status. These implications have to be considered in the light of the prevalence, or pretest probability, of cervical infection in the population group of interest and the post-test probability, or predictive value, of the algorithm.

Positive implications of treating cervical infections (true positives, sensitivity)
· High likelihood of curing infections and avoiding associated symptoms, complications and sequelae as well as of avoiding further STI transmission by infected women.
· Opportunities to expose multiple transmitters (women and/or sex partners) to treatment and preventive intervention as well as to offer couples counselling for STI prevention.
· Opportunity to screen for gender-related risk factors such as violence against women.
· Possibility of enhancing credibility of provider and bolstering education on STD prevention; of improving confidence of patient and community in provider and health care system; and of improving provider's confidence in case management strategy.

Negative implications of treating cervical infections (true positives, sensitivity)
· Adverse drug effects.
· Possible failure to cure infections.
· Potential for problems in couples or social problems.
· Contribution to local antimicrobial resistance.
· Possibility of change in vaginal, oral and bowel flora and of inducing vulvovaginal candidiasis.
· Cost of drugs.

Positive implications of not treating women who do not have cervical infections (true negatives, specificity)
· Avoidance of adverse drug effects.
· No contribution to local antimicrobial resistance.
· Psychological benefit to patient and her partner or partners.
· Avoidance of iatrogenic changes in vaginal, oral and bowel flora.

Negative implications of not treating women who do not have cervical infections (true negatives, specificity)
· Patient may expect treatment and perceive care received as suboptimal.

Positive implications of missed cervical infections (false negatives, lack of sensitivity)
· Possibility of infection being cured without treatment.
· Avoidance of a source of potential couple and social problems as well as of adverse drug effects.
· No contribution to antimicrobial resistance.
· Avoidance of iatrogenic changes in vaginal, oral and bowel flora.

Negative implications of missed cervical infections (false negatives, lack of sensitivity)
· Symptoms may persist.
· If symptoms persist: credibility of provider may be weakened and education on STI prevention may be undermined; confidence of patients and community in provider and health care system may be undermined; provider's confidence in case management strategy may be undermined.
· Probability of sequelae and complications (pelvic inflammatory disease, infertility, ectopic pregnancy).
· Allows continued transmission of infection in community (dynamic impact on STI epidemic).
· Missed opportunities to offer couples counselling for STI prevention and to screen for gender-related risk factors such as violence against women.

Positive implications of unnecessary treatment for cervical infections (false positives, lack of specificity)
· Possibility of treatment of existent urinary tract or other infection.
· May have beneficial effect on pregnancy outcome (12, 13).
· May bolster confidence in provider and health care system.

Negative implications of unnecessary treatment for cervical infections (false positives, lack of specificity)
· Cost of treatment, including opportunity cost.
· Possibility of change in vaginal, oral and bowel flora; may induce vulvovaginal candidiasis and potentially increase susceptibility to STIs.
· May cause adverse drug effects.
· Contributes to local antimicrobial resistance.
· May cause couple and social problems.
· May undermine credibility and confidence in provider and health care system in mutually monogamous couples.

 

Other considerations

Appraising the positive and negative implications of providing or withholding treatment to patients according to a particular decision model and the true infection status is necessary but not sufficient for decision-making purposes. Among other matters that have to be considered in relation to implementing decisions are the stability of risk factors with regard to time, space and the implementer, acceptability by stakeholders, and environmental constraints.

The cross-sectional study design that is typically pursued allows the exploration of risk factors for STIs at one point in time. At the individual level, however, STI risk factors may evolve considerably. For instance, clinical signs, symptoms and markers may change with increasing HIV and AIDS prevalence. Risk factors may evolve in the context of preventive interventions. In fact, one of the objectives of comprehensive clinical care is to promote safe sexual behaviour. Risk factors are thus time-bound and their performance as a detection tool can be expected to vary over time. Risk factors that capture recent sexual risk behaviour can be expected to be more enduring screening mechanisms for identifying individuals at increased risk for STIs than static indicators such as the age at which sexual activity begins.

The identified risk factors for STIs are also space-bound. The prevalence of STIs in the community is a key determinant, driven by numerous factors at the societal and individual levels. National guidelines are typically developed from data collected at one or a few sites. However, their implementation locally may benefit from a review of related regional data. For example, information on sociodemographic factors, health and health-seeking behaviour from a recent census may facilitate the estimation of possible regional differences in the risk factors of interest.

In addition to addressing the time and space attributes, the risk factors must also be acceptable. The content or nature of the identified risk factors may pose acceptability issues for patients and providers. For example, asking a care-seeking woman if she has had a new sex partner in the preceding three months may be unacceptable in a Muslim country (5). The inclusion of specific races or ethnicities as targeted high-risk or low-risk populations may be problematic. Syndromic treatment based on patient history without a genital examination may be perceived by patients as poor care. A recent community-based study in the USA found that physical examinations were perceived by persons seeking STI care and other community members as more accurate than non-invasive urine testing for STIs (14). Patients' perceptions and acceptability issues may influence compliance with treatment, the effect of counselling on prevention, and future STI care-seeking behaviour.

Care providers may also place difficulties in the way of implementation. They may be reluctant to use long lists of risk factors or complex formulas to calculate risk scores. Physicians may view decision- making algorithms as an affront to their clinical judgement. The current case management practices and the extent of change implicit in the proposed new approach may affect the implementation of the new risk assessment. Such practices should be considered in connection with the selection of decision models as well as in implementation policy.

The use of risk factors may pose reproducibility and stability problems in different implementation settings and with different types of providers. Clinical signs that cannot be assessed for practical reasons in some contexts or that may be subject to important measurement error should be avoided, e.g. the assessment of endocervical mucopus by paramedical personnel in peripheral dispensaries. In Madagascar, for example, national syndromic STI treatment guidelines include one algorithm for the vaginal discharge syndrome which relies on a speculum examination, and another algorithm for use in settings where pelvic examinations cannot be reliably performed.

Appraising the nature and severity of the main environmental constraints is central to decision-making. The need to manage the vaginal discharge syndrome by means of algorithms stems directly from environmental constraints, i.e. the inadequacy of financial, human and infrastructural resources, prohibiting appropriate laboratory diagnosis. Specific environmental constraints determine the degree to which overtreatment may be preferred to undertreatment, or the option of using risk assessment as a screening tool for laboratory diagnosis. In Jamaica, algorithms for the management of vaginal discharge were evaluated in patients attending an STI clinic in 1994. Using the results of this study a group of researchers and clinicians selected the algorithm that relied on a pelvic examination and had the highest sensitivity (85%) for detecting cervical infections while accepting a large proportion of overtreatment (specificity 40%) (4).

 

Selection process

The positive and negative implications of false- and true-positives and of false- and true-negatives illustrate the complexity and uncertainty that these factors bring to the decision-making process. In a full rational decision-making approach, probabilities would have to be assigned to these positive and negative implications, and weights would have to be developed in order to express their relative importance. In addition, the factors mentioned above relating to the provider, patient, health care system and implementation would have to be integrated into the choice of the model.

The selection process of a national clinical policy for management of vaginal discharge syndrome in symptomatic women seeking primary care might also include an alternative consisting of no change to the existing clinical practices, allowing investment of the resources required to implement the change in activities deemed to be more cost- effective or desirable for other reasons. Selective presumptive therapy for cervical infection is another alternative that might be worth considering. For example, a strategy of selective presumptive therapy could be designed such that symptomatic women seeking primary care for vaginal discharge syndrome in areas of high STI prevalence would be treated for cervical infection unless they had recently been treated (in accordance with a defined time window) and reported full compliance, or unless they reported no recent sexual behavioural risk (in accordance with defined criteria). Such a strategy could have the advantage of allowing community prevalence to be the prominent factor for patient and partner treatment and preventive education, thereby possibly minimizing couple and social problems and optimizing STI prevention.

Majone argued that the danger of the "internally consistent doctrine" of decisionism, where a few actors engage in deliberately selecting one of the explicitly generated alternatives, is "to produce an overintellectualized version of policy analysis which gives undue emphasis to the more technical aspects of a subject that in fact should be concerned with the whole of the policy process" (15). The implementation of national clinical guidelines might benefit substantially from a selection process that actively included a broad constituency of local stakeholders and professional opinion leaders. This might foster a sense of ownership in the selected guidelines and optimize acceptability and implementation, in addition to achieving an overall improvement in the quality of the decision-making process.

Before 1997, various algorithms were used in Madagascar for syndromic STI care in the absence of adequate local data. A few key players decided that local research was necessary in order to develop national guidelines. A technical advisory committee including researchers, public and private sector decision-makers and clinicians was created to assist with study design. After completion of the field research, the data were reviewed, interpreted, and used by the committee during a three-day workshop in which national guidelines were developed. These have since been widely adopted in the country.

 

Use of algorithms for syndromic treatment of vaginal discharge syndrome

The implementation of policy has been studied by using top-down rational system models (16) based on compelling theory and clear objectives, where resources, control and communications go from policy-makers to implementers. In bottom-up analyses (17), the realities of the actors at street level are taken into account and the role played by physicians and other professionals, when they are deliverers of policy, is taken into consideration. In the bottom-up model, implementation includes negotiation and consensus-building. Implementation has been studied as a combination of these two approaches, e.g. framing implementation as a learning process (18). All of the models, with their sometimes conflicting embedded values, can be helpful as they allow insights from different perspectives and thus reveal the multiple dimensions and challenges of implementation (8). The implementation of syndromic STI treatment in settings where resources are scarce is usually planned as, and may be inherently biased towards, a top-down approach. However, the numerous and persistent difficulties of policy implementation in these complex and highly challenging contexts warrants the use of other frameworks of analysis, including the conceptualization of implementation as an ongoing learning process.

Among the key implementation issues is the training of health care personnel, adapted to the type of provider and clinical experience and taking account of the perceptions, attitudes and current behaviour of providers and patients. Importantly, the framing of implementation as an intervention to change specific behaviours of providers, as often happens, may lead to insufficient attention to other components critical to successful implementation, such as creating and sustaining an enabling environment. For example, the successful implementation of syndromic STI care requires the availability of and access to antibiotics. Policy implementation is also likely to benefit from appropriate supervision mechanisms.

Implementation should include plans for continuing evaluation of the effectiveness and consequences of decision-making. Changes in behavioural or biological parameters necessitate ongoing operations research. The social organization of sexuality may change over time, as may other societal factors that influence the prevalence of STIs and sexual behaviour. Moreover, the implementation of syndromic STI management is typically part of larger efforts to control and prevent STIs. The successful implementation of syndromic STI care as well as other interventions should lead to a decrease in individual high-risk sexual behaviours and in the prevalence and incidence of STIs. Decision models should therefore be adapted to changes in the determinants of STIs. Furthermore, the implementation of decision models for the treatment of the vaginal discharge syndrome should incorporate evaluation of the outcome, i.e. the impact on the prevalence of STIs and on the spread of HIV in the community, as well as of the process. Changes in the prevalence of STIs can be expected to directly influence the performance of the decision model.

The selection of a decision model for the management of the vaginal discharge syndrome therefore typically entails considerable trade-offs and decision-making that generally requires judgements going beyond the mere reasoned use of facts and figures. Empirically developed decision models should be considered as a basis for discourse on policy and should be evaluated together with the evidence, alternatives and arguments by the stakeholders (19).

 

Acknowledgements

We thank Dr Elizabeth Bradley for reviewing and supporting earlier versions of this work.

Conflicts of interest: none declared.

 

 


Résumé

Traitement syndromique des infections à gonocoques et à Chlamydia chez les femmes consultant un service de soins de santé primaires pour des pertes vaginales

Le traitement syndromique des infections à gonocoques et à Chlamydia chez les femmes consultant un service de soins de santé primaires dans un contexte de ressources limitées, tel qu'il est recommandé par l'OMS et mis en œuvre dans de nombreux pays en développement, suppose que l'on trouve un juste milieu entre sous-traitement et surtraitement. Le présent article identifie les facteurs entrant en ligne de compte dans le choix de stratégies adaptées au traitement syndromique dans un tel contexte. En font partie les aspects généraux de la prise de décision et les inconvénients de l'approche rationnelle. L'article expose les aspects positifs et négatifs du traitement ou de l'abstention thérapeutique cidés selon un algorithme spécifique permettant de détecter une infection avec une certaine exactitude (sensibilité, spécificité et valeur prédictive). Les autres aspects décisionnels reconnus sont liés à la mise en œuvre de la décision – stabilité des facteurs de risque en fonction du temps, du lieu et du responsable de la mise en œuvre, acceptabilité par les parties intéressées et contraintes liées à l'environnement. Il est nécessaire d'examiner comme base de discussion les algorithmes de traitement établis selon des critères empiriques, et de les évaluer conjointement avec les faits, les options et les arguments présentés par les divers partenaires.


Resumen

Tratamiento sindrómico en el nivel de atención primaria de las infecciones por gonococos y clamidias en mujeres con flujo genital

El tratamiento sindrómico de las infecciones por gonococos y clamidias en las mujeres que recurren a los servicios de atención primaria en dispensarios con recursos escasos, según las recomendaciones de la OMS que aplican muchos países en desarrollo, obliga a llegar a un compromiso entre el riesgo de sobretratamiento y el de subtratamiento. En este artículo se identifican diversos factores de interés para seleccionar estrategias específicas de tratamiento sindrómico en las mencionadas circunstancias. Se incluyen entre ellos aspectos generales de la adopción de decisiones y salvedades concernientes al enfoque de la adopción de decisiones racionales. Se exponen sucintamente las ventajas y los inconvenientes que se derivan de proporcionar o no tratamiento según los resultados de un algoritmo específico que permite detectar la infección con una precisión determinada, esto es, con una sensibilidad, una especificidad y un valor predictivo determinados. Se hacen también diversas consideraciones sobre la aplicación de los algoritmos decisionales, en relación concretamente con la estabilidad de los factores de riesgo en función del tiempo, del espacio y de los encargados de aplicarlos, con su aceptabilidad por los interesados y con las limitaciones ambientales. Los algoritmos terapéuticos establecidos empíricamen deben constituir la base de las propuestas normativas en este campo, y han de ser evaluados contrastándolos con la evidencia y las alternativas disponibles y con los argumentos de las partes interesadas.


 

 

References

1. Management of sexually transmitted diseases. Geneva, World Health Organization, 1994 (unpublished document WHO/ GPA/TEM/94.1).        [ Links ]

2. Grosskurth H et al. Impact of improved treatment of sexually transmitted diseases on HIV infection in rural Tanzania: randomised controlled trial. Lancet, 1995, 346: 530–536.        [ Links ]

3. Vuylsteke B et al. Clinical algorithms for the screening of women for gonococcal and chlamydial infection: evaluation of pregnant women and prostitutes in Zaire. Clinical Infectious Diseases, 1993, 17: 82–88.        [ Links ]

4. Behets FM et al. Management of vaginal discharge in women treated at a Jamaican sexually transmitted disease clinic: use of diagnostic algorithms versus laboratory testing. Clinical Infectious Diseases, 1995, 21: 1450–1455.        [ Links ]

5. Ryan C et al. Reproductive tract infections in primary health care, family planning, and dermatovenereology clinics: evaluation of syndromic management in Morocco. Sexually Transmitted Infections, 1998, 74 (Suppl. 1): S95–S105.        [ Links ]

6. Diallo MO et al. Evaluation of simple diagnostic algorithms for Neisseria gonorrhoeae and Chlamydia trachomatis cervical infections in female sex workers in Abidjan, Côte d'Ivoire. Sexually Transmitted Infections, 1998, 74 (Suppl. 1): S106– S111.        [ Links ]

7. Simon HA. Reason in human affairs. Palo Alto, CA, Stanford University Press, 1983.        [ Links ]

8. Parsons W. Public policy – an introduction to the theory and practice of policy analysis. Cambridge, Cambridge University Press, 1995.         [ Links ]

9. Simon HA. Models of man: social and rational. New York, John Wiley, 1957.        [ Links ]

10. Stone D. Policy paradox — the art of political decision making. New York, W. W. Norton and Company, 1997.        [ Links ]

11. Kaufinan S. Decision making and conflict management processes in local government. In: Bingham RD et al., eds. Managing local government. Newbury Park, CA, Sage, 1991.        [ Links ]

12. Wabwire-Mangen F et al. Randomized trial of STD treatment during pregnancy: effects on pregnancy outcomes and maternal HIV incidence. Paper presented at: Thirteenth Meeting of the International Society for Sexually Transmitted Diseases Research, Denver, CO, 1999.         [ Links ]

13. Temmerman M et al. Mass antimicrobial treatment in pregnancy. A randomized, placebo-controlled trial in a population with high rates of sexually transmitted diseases. Journal of Reproductive Medicine, 1995, 40: 176–180.        [ Links ]

14. Smith LV et al. The acceptance of urine-based tests vs. physical examinations in the screening of STDs: the community's response. Paper presented at: Thirteenth Meeting of the International Society for Sexually Transmitted Diseases Research, Denver, CO, 1999.         [ Links ]

15. Majone G. Evidence, argument, and persuasion in the policy process. Chelsea, MI, Book Crafters, 1989.        [ Links ]

16. Van Meter D, Van Horn C. The policy implementation process: a conceptual framework. Administration and Society, 1975, 6: 445–488.        [ Links ]

17. Lipsky M. Street-level bureaucracy and the analysis of urban reform. Urban Affairs Quarterly, 1971, 19: 391–409.         [ Links ]

18. Browne A, Wildawsky A. Should evaluation become implementation? In: Pressman J, Wildawsky A, eds. Implementation. Berkeley, CA, University of California Press, 1984.        [ Links ]

19. Lasswell HD. The technique of decision seminars. Midwest Journal of Political Science, 1960, 4: 213–236.        [ Links ]

 

 

1 Department of Medicine and Department of Epidemiology, School of Public Health, University of North Carolina, Chapel Hill, NC, USA. Correspondence should be addressed to Dr Behets at the following address: 2104G McGavran-Greenberg Hall, CB 7400, Chapel Hill, NC 27599–7400, USA (email: frieda_behets@unc.edu).

Ref. No. 00-0716