Health information technology in primary health care in developing countries: a literature review

Technologies de l'information et soins de santé primaires dans les pays en développement : une revue de la littérature

Tecnologías de la información sanitaria en la atención primaria en los países en desarrollo: una revisión de la literatura

Elaine Tomasi¹ ; Luiz Augusto Facchini; Maria de Fatima Santos Maia

ABSTRACT

This paper explores the debate and initiatives concerning the use of information technology (IT) in primary health care in developing countries. The literature from 1992–2002 was identified from searches of the MEDLINE, Latin American and Caribbean Health Science Literature Database (LILACS), Cochrane Library and Web of Science databases. The search identified 884 references, 350 of which were classified according to the scheme described by the Pan American Health Organization (PAHO). For the analysis of advantages, problems and perspectives of IT applications and systems, 52 articles were selected according to their potential contribution to the primary health-care processes in non-developed countries. These included: 10 on electronic patient registries (EPR), 22 on process and programmatic action evaluation and management systems (PPAEM) and 20 on clinical decision-support systems (CDS). The main advantages, limitations and perspectives are discussed.

Keywords: Primary health care; Information technology; Information systems/utilization; Decision support systems, Clinical/utilization; Medical records systems, Computerized/utilization; Education, Distance; Telemedicine/utilization; Review literature; Developing countries (source: MeSH, NLM).

RÉSUMÉ

Le présent article examine les débats et les initiatives concernant l'utilisation des technologies de l'information dans le cadre des soins de santé primaires dans les pays en développement. Les articles publiés en 1992-2002 ont été identifiés au moyen d'une recherche dans plusieurs bases de données - MEDLINE, LILACS (Latin American and Caribbean Health Science Literature Database), Cochrane Library et Web of Science. La recherche a fourni 884 références, dont 350 ont été classées selon le système décrit par l'Organisation panaméricaine de la Santé. Pour l'analyse des avantages, des limites et des perspectives des applications et systèmes informatiques, 52 articles ont été sélectionnés pour leur contribution potentielle aux processus de soins de santé primaires dans les pays non développés : 10 concernaient les dossiers médicaux informatisés, 22 les systèmes d'évaluation et de gestion des processus et de l'action programmatique et 20 les systèmes d'aide à la décision clinique. L'article présente une discussion des principaux avantages, limites et perspectives de ces applications et systèmes.

Mots clés: Soins santé primaire; Technologie de l'information; Système information/utilisation; Système informatique aide décision clinique/utilisation; Dossier médical informatisé/utilisation; Enseignement à distance; Télémédecine/utilisation; Revue de la littérature; Pays en développement (source: MeSH, INSERM).

RESUMEN

Palabras clave: Atención primaria de salud; Tecnología de la información; Sistemas de información/utilización; Sistemas de apoyo a decisiones clínicas/utilización; Sistemas de registros médicos computarizados/utilización; Educación a distancia; Telemedicina/utilización; Literatura de revisión; Países en desarrollo (fuente: DeCS, BIREME).

Introduction

In health care, especially in developed countries, the use of different types of information technology (IT) has progressed considerably since the beginning of electronic patient registration, leading to improvements in the interfacing and fusion capabilities of a large variety of computer and telecommunication technologies. Such evolution can be attributed partly to the peculiarities of the health-care sector — namely, its scope, its status as a large market for computer businesses, and its need for facilities for information storage and management, improvements in quality of care, and expenditure control, in both the public and private sectors.

The health-care systems of developed countries have generally been in existence for at least two decades longer than those in developing countries and have been accumulating experience in the use of such technologies, especially in primary health care, which has provided relevant lessons regarding the use of IT in the health-care system as a whole. The computerization of medical records in hospitals and health clinics; the use of the Internet for communication and information exchange; the development of magnetic cards for user identification; electronic scheduling systems for appointments, examinations and hospital admissions; and computerized protocols for diagnosis and treatment support are just a few examples. Health IT has facilitated access to health literature, both to online journals, books and databases, and offline to informational CD-ROMs, that support practising professionals.

A recent review of the literature on the computerization process in basic health care between 1980 and 1997 summarizes in its title the current situation, i.e. "a descriptive feast but evaluative famine" (1). The authors pointed out the lack of research on the impact of IT on the health status of the population, and the methodological limitations in the design of the studies published so far.

In Brazil, as in other developing countries, structural deficiencies due to the current economic situation have led to considerable deficits in social policies — including those related to public health care. Changes in demographic and epidemiological profiles, in urbanization and in the level of industrialization have created a need for new models of health care. Such models attribute an increasing level of importance to primary health care, the strengthening of which is considered central to the improvement of health-care coverage (2, 3).

Methodology

Identification of publications

Publications were identified by an initial generic search using words from any database field (i.e. words from titles, keywords and abstracts) after which all keywords yielding relevant publications were listed. Some of the search terms used were:

• database management systems AND primary health care;
• health information AND primary health care;
• IT AND primary health care;
• information systems AND primary health care;
• software AND primary health care; and
]]> software AND ambulatory care information systems.

Searches were performed for publication dates ranging from 1992 to 2002, and data sources included MEDLINE, Latin American and Caribbean Health Science Literature Database (LILACS), Cochrane Library and Web of Science. Some additional relevant studies were identified using a demonstration version of EMBASE and from the web site www.hi-europe.info.

Inclusion and exclusion criteria

Duplicate references were excluded, as were references without abstracts, those not specifically related to health, and those that were not concerned with health IT. Publications with abstracts were classified according to the PAHO criteria (Table 1) (4). From the PAHO classification, the three categories most closely related to primary health-care practices were selected, i.e. systems for facilities, decision-support systems and electronic patient registration. The decision-support systems were subdivided into two groups: clinical decision-support (CDS) systems and process and programmatic action evaluation and management (PPAEM) systems. The systems initially considered as belonging to the facility category were later reclassified as belonging either to electronic patient registries (EPR) or to PPAEM.

A further selection of abstracts was performed, including those related to the development and/or evaluation of IT within the scope of primary health care, which emphasized their potential contribution to health-care evaluations. Papers included at this stage were assigned to the following categories: benefits, barriers to implementation and improvement requisites.

Results

The search yielded a total of 884 publications (Fig. 1). Of these, 534 (60.4%) were excluded (references that did not include both an abstract and full text (331); those that were duplicated between databases (14); those with no mention of any kind of health IT, beyond opinion articles, editorials and essays on generic issues related to the subject (139); and publications dealing solely with technical specifications for the writing of software, not necessarily for the health sector (50)).

According to the PAHO classification (Table 1), most of the publications identified (27.7%) described the development and/or implementation of information systems at health facilities, including hospitals, clinics, physician's offices and diagnosis and treatment support centres. These information systems were used for data storage, processing, recovery or diffusion purposes. This group also included articles about systems designed for the management of clinical and administrative information within a specific facility or between different facilities.

The final group comprised publications related to telemedicine such as support systems for diagnosis teleconferences, transmission of high-resolution images and vital signs for long-distance diagnosis and robotic telesurgery. These subjects accounted for 3.1% of the publications, and medical imaging systems designed to store, process, recover and transmit medical images for 2.6%.

Following a detailed analysis of the 350 abstracts included in Table 1, 135 were excluded. These were publications concerned with data exchange, educational activities, and telemedicine or medical imaging, or because of language problems (three). Of the 215 remaining references, 163 were not concerned with the evaluation of technology (Fig. 1).

We attempted to obtain the full texts of the remaining 52 publications: 45 of them were successfully located, corresponding to 13% of the total. We then proceeded to the identification of advantages, problems and potential solutions related to the use of computerized systems in primary health care. From the reference lists included in the articles, and further searches using other sources, seven more relevant publications were located, giving a total of 52.

Electronic patient registries

When compared to manual registration, the main advantages of electronic patient registries (EPR) are greater accuracy (5) and a higher proportion of correct information (6, 7); time saved in locating information (8); more economical use of financial resources; and greater ease and speed of recovery of patient data (12).

Several articles reported on the limitations of such technologies, highlighting the resistance and difficulties of using EPR among health-care professionals, especially physicians (13–15). Emphasis was also placed on aspects related to confidentiality of information and respect for privacy, the need for continuing training and support for human resources (16, 17), and the lack of automatic standardization and codification of the data entered (14).

Table 02

Process and programmatic action evaluation and management systems

The first group of process and programmatic action evaluation and management (PPAEM) systems identified were those concerning patient referral and "counter-referral activities" (i.e. the return of the patient to his or her physician after specialist consultation) both between different levels of care (e.g. to specialists or hospitals) and, for example, for the electronic return of the results of laboratory examinations. Their main advantages are reliability (7), speed (8) and the optimization of available resources (7).

The second group of applications were those designed for the monitoring of patients linked to specific health programmes, such as immunization at mother and child clinics, antenatal care and diabetes programmes. This monitoring was mainly carried out by means of "notices" generated when patients missed scheduled appointments, and the issuing of pre-appointment reminders. The advantages reported included reductions in registration errors, identification of absentees, integration of prevention and control activities, and detection of risk factors and complications (8, 20–22).

A third set of publications was concerned with the analysis and extraction of selected information from electronic patient registries, allowing the identification of risk factors and groups of at-risk patients and the obtainment of care-quality indicators and their comparison between different health units (23, 24). The authors agreed that such systems could assist with evaluations of morbidity and patterns of drug prescription (25), allow managers to monitor compliance with conduct and norms regulated between different levels of care (26), and optimize the prevention and early detection of risk factors (27).

Table 03

One limitation is the lack of studies evaluating the impact of the use of these systems on quality of care (11). Another drawback is the lack of standardization among the different systems which reduces the usefulness of automatically generated indicators (28). When data entry is retrospective, there is a tendency to transfer the deficiencies of a manual registry to the computerized registry (29). It is often necessary to develop additional system tools, such as, for example, codification of the reasons for appointments (30).

Clinical decision-support systems

Problems such as hypertension and cardiopathies in general (31, 32), asthma (33–35), and depression (36, 37) are among the most cited examples of clinical decision-support (CDS) systems. Such health problems, together with prevention programmes, constitute the main reasons for utilization of primary health care, making the adoption of standardized protocols that can be optimized with the support of IT easier (34). Positive experiences have been reported to result from implementation of these systems (38), including increases in physician adherence to standardized therapeutic plans (32, 33, 39), cost reduction (33), and easier standardization and regulation of requests for secondary and tertiary health care and for examinations (40), thus reducing variability between services.

From an administrative point of view, it is possible to obtain greater adhesion to public policies (41). Standardized programmes for the early detection of diseases would tend to have greater diagnostic value thus contributing towards the promotion of equity, and the reduction of complications and costs related to more complex treatments (42).

Table 04

As with the other technologies reviewed, the limitations were related to the low adhesion rates among health-care professionals, the great variety of systems available which hindered evaluation of their validity and reproducibility, and difficulties in standardization and integration with other applications (43).

Discussion

Given the rapid progress in health IT in developed countries and in spite of the differences in infrastructure and health facilities, it is useful to extract some relevant lessons for developing countries, especially those that are trying to design health informatics policies for primary care services. MEDLINE, LILACS, the Cochrane Library and the Web of Science were the main sources of published papers, which mainly reported the experiences of developed countries. One possible gap in the present review is related to the fact that papers from non-indexed journals (i.e. those not registered in international databases, such as MEDLINE) could not be identified. We tried to minimize this potential bias by including both papers that reported the experiences in developed countries and experiences in developing countries.

There is a consensus concerning the usefulness of computerized systems in primary health care, especially for promoting greater efficiency in management processes. Although studies evaluating the impact of such technologies on indicators of health and quality of care are still rare, most authors agree that positive effects attributed to the implementation of the different systems and applications can be maintained during routine use and improved through monitoring. With regard to EPR the main lessons are related to system security, especially the maintenance of privacy and confidentiality. The interconnection between different systems and software is another relevant issue. It would be imperative to adopt standards for vocabulary, contents, images, objects and communication tasks.

The finding of a low level of adhesion among physicians to protocols for computerization in primary health care is almost ubiquitous. Although the reasons have yet to be explained, it is possible that the autonomy regarding clinical decisions — a paradigm of traditional medical practice — must be made to coexist with regulated and more cooperative activities, although this will be no easy task. Furthermore, a substantial number of the articles reviewed stressed the need for continued motivation and training for all team members as an important requisite for the success of any initiative in this area (19). This lesson would be very relevant to the establishment of IT in primary health-care systems. It may be pertinent here to quote the reflections by Branco (2) on the significance of training, that is, the amplification of knowledge: "… knowledge of the logic behind health information production and flux must be provided to all persons involved, and should include the understanding of the goals of the systems to which they have access, and of the utilization possibilities of the information produced …"

Another consensual aspect was the difficulty of finding adequate methods for evaluation, given the enormous variety of applications and contexts in which IT is used. Similarly to the situation for medicinal drugs some 40 years ago, IT has not yet been regulated to ensure its safety and efficacy. Thus, every facility or organization, in order to fulfil its particular needs, "orders" specific products to provide a solution to specific problems, reproducing the specialists' view of health in their administrative and evaluative demands. The results of specific evaluations lack external validation, because health services are extremely variable in terms of population seen, team composition, qualifications, motivation and extent of computerization. This hampers comparability and generalizability (37). In addition, the complexity of clinical and organizational management processes is often underestimated (44).

In the consideration of CDS systems in particular, emphasis has been placed on quality and safety concerns. The main drawbacks of such systems include the lack of consensual standardization for a number of conditions, the probably negative effect on the physician–patient relationship (for example, the perception that computers take over the physician's role), the difficulty in addressing complex conditions, the profusion of different systems with different formats, and the need for training and support (1).

The most important lesson comes from the trends in adoption of national health information systems. Countries such as Australia, Canada and England have recently been heavily involved in implementing large information systems in an unprecedented effort towards standardization and the incorporation of new technologies (48–50). Scotland is another example; it maintains a single information system that functions in 75% of its services, thus facilitating the comparison of data and the extension of benefits resulting from its improvement (26). Brazil is currently planning a national health information policy that should lead to a significant improvement in public health care, especially in primary health-care services. If the developing countries learnt the important lessons provided by the developed countries, they would be able to reduce the time and resources required to increase IT utilization.

Conflicts of interest: none declared.

References

1. Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97. BMJ 2001;322:279-82.        [ Links ]

2. Branco MAF. Informação e tecnologia: desafios para a implantação da Rede Nacional de Informações em Saúde. [Information and technology: challenges to develop a national health information network.] Physis: Rev Saúde Coletiva 1998;8:95-123. In Portuguese.        [ Links ]

3. Bodstein R. Atenção básica na agenda da saúde. [Primary care in the health agenda.] Ciênc Saúde Coletiva 2002;7:401-12. In Portuguese.        [ Links ]

4. Organización Panamericana de la Salud. Sistemas de información y tecnologia de información en salud: desafios y soluciones para América Latina y el Caribe. [Health information systems and technologies: challenges and perspectives to Latin America and the Caribbean.] Washington: Pan American Health Organization; 1998. In Spanish.        [ Links ]

5. Hassey A, Gerrett D, Wilson A. A survey of validity and utility of electronic patient records in a general practice. BMJ 2001;322:1401-5.        [ Links ]

6. Flygt C, Homelius B, Jacobsson B, Foldevi M, Trell E. Essential data set for computer management of distributed primary care services. Medical Informatics 1995;20:331-41.        [ Links ]

7. Gaudet LA. Electronic referrals and data sharing: can it work for health care and social service providers? Journal of Case Management 1996;5:72-7.        [ Links ]

8. Borowitz SM. Impact of a computerized patient tracking system in a pediatric clinic. International Journal of Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium Medical Informatics 1996. p. 508-11.        [ Links ]

9. Ornstein SM, Garr DR, Jenkins RG. A comprehensive microcomputer-based medical records system with sophisticated preventive services features for the family physician. Journal of the American Board of Family Practice 1993;6:55-60.        [ Links ]

10. Singh AK, Kohli S, Moidu K, Bostrom K, Trell E, Wigertz O. Primary health care computing analysis of Swedish maternal health records. Journal of Medical Systems 1994;18:221-8.        [ Links ]

11. Branger PJ, van't Hooft A, van der Wouden JC, Moorman PW, van Bemmel JH. Shared care for diabetes: supporting communication between primary and secondary care. International Journal of Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium Medical Informatics 1999;53:133-42.        [ Links ]

12. Wager KA, Lee FW, White AW, Ward DM, Ornstein SM. Impact of an electronic medical record system on community-based primary care practices. Journal of the American Board of Family Practice 2000;13:338-48.        [ Links ]

13. Manchester GW, Raia TJ 3rd, Scott J, Emery L, Russo AR. Primary Care Health Information System: a hybrid electronic–paper medical record system. Journal of Ambulatory Care Management 1992;15:13-29.        [ Links ]

14. Chin HL, Krall MA. Successful implementation of a comprehensive computer-based patient record system in Kaiser Permanente Northwest: strategy and experience. Effective Clinical Practice 1998;1:51-60.        [ Links ]

15. Souther E. Implementation of the electronic medical record: the team approach. Computers in Nursing 2001;19:47-55.        [ Links ]

16. Blignaut PJ, McDonald T, Tolmie CJ. System requirements for a computerised patient record information system at a busy primary health care clinic. Curationis 2001;24:68-76.        [ Links ]

17. Williamson L, Stoops N, Heywood A. Developing a district health information system in South Africa: a social process or technical solution? Medinfo 2001;10:773-7.        [ Links ]

18. Mangoud AM, Al-Ruwashed FA. A computer program to fit a family and community medicine set-up. Computer Methods and Programs in Biomedicine 2002;68:129-34.        [ Links ]

19. Nobel J. Changes in health care: challenges for information system design. International Journal of Bio-medical Computing 1995;39:35-40.        [ Links ]

20. Singh AK, Kohli M, Trell E, Wigertz O, Kohli S. Bhorugram (India): revisited. A 4 year follow-up of a computer-based information system for distributed MCH services. International Journal of Medical Informatics 1997;44:117-25.        [ Links ]

21. Szecsenyi J, Uphoff H, Ley S, Brede HD. Influenza surveillance: experiences from establishing a sentinel surveillance system in Germany. Journal of Epidemiology and Community Health 1995;49 Suppl:1:9-13.        [ Links ]

22. Wei F, Wright K, Heaton T, Kincaid W. A regional integrated information system to assure maternal-child health services: a progress report. American Journal of Preventive Medicine 1996;12 Suppl:20-5.        [ Links ]

23. Adams WG, Conners WP, Mann AM, Palfrey S. Immunization entry at the point of service improves quality, saves time, and is well-accepted. Pediatrics 2000;106:489-92.        [ Links ]

24. Assaf AR, Banspach SW, Lasater TM, Ramsey J, Tidwell RJ, Carleton RA. The FPbase microcomputer system for managing community health screening and intervention data bases. Public Health Reports 1992;107:695-700.        [ Links ]

25. Wilson AE, Pollock C, Weekes T, Dowell A. Can general practice provide useful information? Evaluation of a primary health care information project in northern England. Journal of Epidemiology and Community Health 1995;49:227-30.        [ Links ]

26. Milne RM, Taylor MW, Taylor RJ. Audit of populations in general practice: the creation of a national resource for the study of morbidity in Scottish general practice. Journal of Epidemiology and Community Health 1998;52 Suppl 1:20S-24S.        [ Links ]

27. Ivankovic D, Kern J, Bartolic A, Vuletic S. Analysis of data as information: quality assurance approach. International Journal of Health Care Quality Assurance 1993;6:9-12.        [ Links ]

28. Rascon-Pacheco RA, Santillana-Macedo MA, Romero-Arredondo ME, Rivera-Icedo BM, Romero-Cancio JA, Cota-Rembau AI. Epidemiologic monitoring system for the diabetic patient; use of computerized technology in the quality of medical care. Salud Publica de Mexico 2000;42:324-32.        [ Links ]

29. McColl A, Roderick P, Smith H, Wilkinson E, Moore M, Exworthy M, et al. Clinical governance in primary care groups: the feasibility of deriving evidence-based performance indicators. Quality in Health Care 2000;9:90-7.        [ Links ]

30. Letrilliart L, Viboud C, Boelle PY, Flahault A. Automatic coding of reasons for hospital referral from general medicine free-text reports. International Journal of Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium 2000. p. 487-91.        [ Links ]

31. Pinciroli F, Combi C, Pozzi G, Rossi R. MS2/Cardio: towards a multi-service medical software for cardiology. Methods of Information in Medicine 1992;31:18-27.        [ Links ]

32. Starkey C, Michaelis J, de Lusignan S. Computerised systematic secondary prevention in ischaemic heart disease: a study in one practice. Public Health 2000;114:169-75.        [ Links ]

33. Austin T, Iliffe S, Leaning M, Modell M. A prototype computer decision support system for the management of asthma. Journal of Medical Systems 1996;20:45-55.        [ Links ]

34. Khoury AT. Support of quality and business goals by an ambulatory automated medical record system in Kaiser Permanente of Ohio. Effective Clinical Practice 1998;1:73-82.        [ Links ]

35. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ 2002;325:941.        [ Links ]

36. Kobak KA, Taylor LH, Dottl SL, Greist JH, Jefferson JW, Burroughs D, et al. A computer-administered telephone interview to identify mental disorders. JAMA 1997;278:905-10.        [ Links ]

37. Unutzer J, Katon W, Williams JW Jr, Callahan CM, Harpole L, Hunkeler EM, et al. Improving primary care for depression in late life: the design of a multicenter randomized trial. Medical Care 2001;39:785-99.        [ Links ]

38. Bradley JH, King DE. Electronic medical records for prenatal patients: challenges and solutions. MD Computing: Computers in Medical Practice 1998;15:316-22, 331.        [ Links ]

39. Papshev D, Peterson AM. Electronic prescribing in ambulatory practice: promises, pitfalls, and potential solutions. American Journal of Managed Care 2001;7:725-36.        [ Links ]

40. Alonso Lopez FA, Anzola Fernandez B, Arratibel Arrondo I, Gancedo Gonzalez Z. The evaluation of a computer program for the filling of repeat prescriptions. Atencion primaria / Sociedad Espanola de Medicina de Familia y Comunitaria 1993;12:96-8.        [ Links ]

41. Chuang CT. An efficient fault-tolerant out-patient order entry system based on special distributed client/server architecture. Medical Informatics 1998;23:145-57.        [ Links ]

42. FitzHenry F, Kiepek WT, Shultz EK, Byrd J, Doran J, Miller RA. Implementing outpatient order entry to support medical necessity using the patient's electronic past medical history. Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium 2002 p. 250-4.        [ Links ]

43. Delaney BC, Fitzmaurice DA, Riaz A, Hobbs FD. Can computerised decision support systems deliver improved quality in primary care? Interview by Abi Berger. BMJ 1999;319:1281.        [ Links ]

44. Cibulskis RE, Posonai E, Karel SG. Initial experience of using a knowledge based system for monitoring immunization services in Papua New Guinea. Journal of Tropical Medicine and Hygiene 1995;98:107-13.        [ Links ]

45. Rigby M, Forsstrom J, Roberts R, Wyatt J. Verifying quality and safety in health informatics services. BMJ 2001;323:552-6.        [ Links ]

46. Heathfield H, Pitty D, Hanka R. Evaluating information technology in health care: barriers and challenges. BMJ 1998;316:1959-61.        [ Links ]

47. Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt. BMJ 2003;326:860-3.        [ Links ]

48. Australian Department of Health and Ageing, National Health Information Management Advisory Council (NHIMAC). Health Online: a health information action plan for Australia; 2003. Available from URL: http://www.health.gov.au/healthonline        [ Links ]

49. Government of Canada. Canada Health Infoway. Toronto, Ontario; 2003. Available from URL: http://www.canadahealthinfoway.ca        [ Links ]

50. Majeed A. Ten ways to improve information technology in the NHS. BMJ 2003;326:202-6.        [ Links ]

Submitted: 30 July 2003 – Final revised version received: 9 May 2004 – Accepted: 19 May 2004

1 Correspondence should be sent to Dr Tomasi (email: etomasi@epidemio-ufpel.org.br).

Additional references for Tables 3 and 4

Af Klercker T, Zetraeus S. Dilemmas in introducing World Wide Web-based information technology in primary care: a focus group study. Family Practice 1998;15:205-10.

Bergvin L, Johansson B, Borjesson U. Distribution of laboratory test results to primary health care centres with the EDIFACT standard. Clinica Chimica Acta 1993;222:141-5.

Gerbert B, Bronstone A, Maurer T, Hofmann R, Berger T. Decision support software to help physicians triage skin cancer: a pilot study. Archives of Dermatology 2000;136:187-92.

Gordon C, Jackson-Smale A, Thomson R. DILEMMA: logic engineering in primary care, shared care and oncology (AIM Project A2005). Computer Methods and Programs in Biomedicine 1994;45:37-9.

Hobbs FD, Delaney BC, Carson A, Kenkre JE. A prospective controlled trial of computerized decision support for lipid management in primary care. Family Practice 1996;13:133-7.

Margolis CZ, Warshawsky SS, Goldman L, Dagan O, Wirtschafter D, Pliskin JS. Computerized algorithms and pediatricians' management of common problems in a community clinic. Academic Medicine 1992;67:282-4.

Ross KM. Complex ambulatory settings demand scheduling systems. Ambulatory Outreach 1998:23-5.

Safran C. Searching for answers on a clinical information system. Methods of Information in Medicine 1995;34:79-84.

Shiffman RN, Brandt CA, Freeman BG. Transition to a computer-based record using scannable, structured encounter forms. Archives of Pediatrics and Adolescent Medicine 1997;151:1247-53.

Tirol R, Menezes PR. Computerized psychiatric interviews: can they help to improve medical care? Revista Paulista de Medicina 1998;116;1821-2.