## Services on Demand

## Article

## Indicators

## Related links

## Share

## Revista de Saúde Pública

##
*On-line version* ISSN 1518-8787

### Rev. Saúde Pública vol.41 n.3 São Paulo Jun. 2007 Epub Mar 29, 2007

**ORIGINAL ARTICLES**

**Data
envelopment analysis for evaluating public hospitals in Brazilian state capitals**

**Antonio C Gonçalves ^{I};
Cláudio P Noronha^{I}; Marcos
PE Lins^{II}; Renan
MVR Almeida^{II}**

^{I}Coordenação
de Indicadores Gerenciais. Secretaria Municipal da Saúde. Rio de Janeiro,
RJ, Brasil

^{II}Programa
de Engenharia da Produção. Coordenação dos Programas
de Pós-graduação de Engenharia (COPPE). Universidade Federal
do Rio de Janeiro. Rio de Janeiro, RJ, Brasil

**ABSTRACT**

**OBJECTIVE:**
To apply the Data Envelopment Analysis (DEA) methodology for evaluating the
performance of public hospitals, in terms of clinical medical admissions.

**METHODS:**
The efficiency of the hospitals was measured according to the performance of
decision-making units in relation to the variables studied for each hospital,
in the year 2000. Data relating to clinical medical admissions in hospitals
within the public system in Brazilian state capitals and Federal District (mortality
rate, mean length of stay, mean cost of stay and disease profile) were analyzed.
The canonical correlation analysis technique was introduced to restrict the
variation range of the variables used. The constant returns to scale model was
used to generate scores that would enable assessment of the efficiency of the
units. From the scores obtained, these cities were classified according to their
relative performance in the variables analyzed. It was sought to correlate between
the classification scores and the exogenous variables of the expenditure on
primary care programs per inhabitant and the human development index for each
state capital.

**RESULTS:**
In the hospitals studied, circulatory diseases were the most prevalent (23.6%
of admissions), and the mortality rate was 10.3% of admissions. Among the 27
state capitals, four reached 100% efficiency (Palmas, Macapá, Teresina
and Goiânia), seven were between 85 and 100%, ten were between 70 and
85% and ten had efficiency of less than 70%.

**CONCLUSIONS:**
The tool utilized was shown to be applicable for evaluating the performance
of public hospitals. It revealed large variations among the Brazilian state
capitals in relation to clinical medical admissions.

**Keywords: **National
Health System (BR). Health services evaluation. Hospital services. Efficiency,
organizational. Information systems. Data analysis.

**INTRODUCTION**

Data envelopment analysis (DEA), which was introduced by
Charnes et al^{3} in 1978 and extended by Banker et
al^{1} (1984), provides a representation of the structure
formed by decision-making units (DMUs), with inputs and outputs
that are defined in such a way as to be able to assess the
relative efficiency of these DMUs. This efficiency is defined
from the observed performance of the DMUs in relation to the
variables analyzed. It is an empirical measurement and not a
theoretical or conceptual reference.^{10,11} This means
that its scores are a comparison measurement that is more
appropriate than the more commonly used indicators (e.g. number
of procedures per time period or mortality rates), which may be
highly dependent on the specific characteristics of a
population.

This method establishes a "common region" on the basis of the
data (variables) of the DMUs, thereby creating an efficiency
index that reflects the importance of each variable for each DMU.
Thus, in the common region, units with behavioral patterns that
are most optimized for these variables are sought. The maximum
value for this index (in each DMU) is then assumed to be an
"empirical maximum" efficiency, from which a relative
classification of the units becomes possible.^{10} From
this, the method also provides "excellent" values that the
variables should attain, for the DMU to be able to move from
"inefficient" to "efficient". DEA has recently been used in the
health sector for establishing reference standards for hospitals,
clinics or health services, particularly in developing
countries.^{3,4,7,9,12,13} In Brazil, one of the rare
studies using this methodology was carried out in 2001, to
compare university and general hospitals in the municipality of
Rio de Janeiro.^{11}

In December 2000,
there were 913 hospitals available to the Brazilian national health system (*Sistema
Único de Saúde* – SUS) in the country's state capitals
(public, university and philanthropic hospitals and those available through
access agreements). During that year, these hospitals were responsible for 742,833
admissions relating to clinical medicine. Methodologies that allow assessment
of these hospitals' performance urgently need to be developed, both because
of the scarcity of resources in the health sector and because users demand and
have a right to a system with quality services.^{14}

The objective of the present study was to apply DEA in studying the efficiency of the a hospital network, using the SUS hospitals in Brazilian state capitals as an example.

**METHODS**

The database was
formed by admissions to SUS hospitals in the country's state capitals in 2000,
and the data were obtained from the SUS hospital information system (Datasus).^{1}
The DEA was performed using the Frontier Analyst Professional software*.*^{2}
The canonical weights, canonical correlation, restriction intervals for the
weights of the variables and the other statistical procedures were generated
in the Statistica software*.*

To comparatively assess the efficiency of the SUS hospitals in the Brazilian state capitals, their admissions in the clinical medical category were analyzed. In addition to clinical medical admissions in the strict sense, admissions in other clinical sub-specialties were also included, such as: cardiology, endocrinology, clinical oncology, infectology and pneumology. The following variables were used:

Inputs: mortality rate (mortality) and mean length of stay in hospital (mean length of stay).

Outputs: percentages of admissions relating to the three chapters of the International Classification of Diseases (ICD) with the greatest mortality percentages, respectively: neoplasias; infectious and parasitic diseases (IPD) and diseases of the circulatory system (circulatory); and mean value paid through the Hospital Admission Authorization (mean HAA).

The DEA used the Constant Returns to Scale (CRS) model, in which efficiency was defined as the ratio of the weighted sum of inputs and outputs, and the objective of the method was to maximize this ratio for each DMU. A unit (capital) that obtained the maximum value for this maximization (1, by definition) was considered to be "efficient" and if not, it was said to be "inefficient" "inefficient" (Annex).

Firstly, a canonical
correlation analysis between the input and output variables was used to identify
restriction intervals for the weights of these variables that were needed for
DEA (Annex).^{2,8,}^{3}
Next, the scores thus obtained were correlated (Pearson's coefficient) with
the exogenous variables "per capita expense of primary healthcare programs"
and "human development index (HDI) for the cities studied", for the year 2000.^{4}

**RESULTS**

Diseases of the circulatory system were prominent, accounting for 23.6% of the admissions in the hospitals studied, with a range from 28.7% in Cuiabá to 8.9% in Macapá. The infectious and parasitic diseases group, which included AIDS and tuberculosis, corresponded to 9.9% of the admissions (maximum of 18.7% in Manaus and minimum of 5.9% in Brasília). The neoplasia group represented 7.5% (maximum of 19.3% in Belo Horizonte and minimum of 0.3% in Aracaju) (Table 1). These three groups totaled 41% of all the admissions within the system.

The mortality rate was 10.3% of the admissions (maximum of 17.6% in Natal and minimum of 4.1% in Macapá). The mean length of stay was 8.8 days (maximum of 12.6 in Rio de Janeiro and Florianópolis, and minimum of 4.7 in Palmas). The mean amount for admission reimbursements via HAAs was R$ 405.34 for all the admissions (maximum of R$ 542.23 in Campo Grande and minimum of R$ 207.90 in Macapá).

Table 2 summarizes the results from the canonical correlation analysis (canonical weights, canonical correlation coefficients and restriction intervals for the weights of the variables). Table 3 shows the classification of the state capitals according to the efficiency attained using DEA, the observed values and the estimated values for minimization of the inputs. Among the 27 state capitals, four achieved 100% efficiency (Palmas, Macapá, Teresina and Goiânia), seven were between 85% and 100%, ten were between 70% and 85% and ten presented less than 70%. Table 3 shows the estimated values for the inputs needed for each capital to achieve 100% efficiency. For example, Rio de Janeiro (66.5% efficiency) has observed values for mortality and mean length of stay of 16.0% and 12.6 days, respectively. In this case, for the city to achieve 100% efficiency, it would be necessary to reduce these rates to the levels of 7.6% and 8.9 days, respectively.

No linear correlation was found between the classification scores and the municipal HDI values (r=0.03; p>0.05), or between the classification scores and expenses per capita (r=0.03; p>0.05).

**DISCUSSION**

Contrary to other studies that utilized the DEA methodology in the field of health sector assessment in Brazil, the present study was restricted to one specific specialty (clinical medicine) and did not cover hospitals as a whole. It was thus sought to ensure that comparisons were made between entities with intrinsically greater homogeneity. For this, classical indicators were used, such as length of stay and mortality rate, and the admissions relating to the three chapters of the ICD with greatest weight in the system.

In the Brazilian public health system, admissions to hospitals in the system are paid for through HAAs. The amounts of these payments depend on the services provided, the technological backup and the materials used, excluding salaries and infrastructure expenditure. In defining the DEA model, the disease profile and the mean amount of the HAA payments were taken to be "fixed", since they represent real demands from affections that are prevalent among the population and the hospital resources at a given time.

Contrary to what is commonly done in developing causal models, in the present study the mortality variable was used as an input to the system, because of the differentiated characteristics of the methodology used. In DEA, the groups of variables called "inputs" and "outputs" are used to generate the factor that is the great differential of the method, i.e. the classification scores resulting from the minimization of the inputs or the maximization of the outputs. In the present study, the form that is considered most natural was used, i.e. minimizing the inputs "mortality rate" and "length of stay". Nonetheless, no methodological or interpretative difference would arise if these were used as outputs. Thus, if the inputs had been considered to be outputs, and vice versa, and the analysis had been undertaken such that outputs were maximized, the same hierarchical classification (the scores) would have been obtained, without introducing any alterations of logic in the results obtained.

The mathematical structure of DEA models often means that a
DMU is considered to be efficient because zero weight is
attributed to variables that are then disregarded in evaluating
the unit. Defining restrictions from the canonical
weights,^{8} as introduced in the present study not only
allows the importance of the variables for DEA to be evaluated,
but also minimizes the quantity of variables with zero weight.
This is an important methodological step, because it avoids
rejecting variables that may be relevant in the process of
forming the efficiency scores. In the original concept for DEA
models (in economics) only "desirable" outputs were considered,
i.e. those for which maximization is of interest (for example,
maximizing production while considering fixed
supplies).^{6} In the present study, the percentages of
admissions relating to the three ICD chapters of greatest weight
and the mean amounts of HAA payments were considered to be
outputs, and the inputs (to be minimized) were the mortality rate
and mean length of stay. The mean amount of HAA payments was used
as a "proxy" for the complexity of the procedures carried out,
and this made it possible to reject the hypothesis that the
results had been influenced by the differentiated levels of
complexity of these procedures.

Some studies^{9} have used DEA to perform economic
assessments on health care units. The present study, however, was
not concerned with economic performance, which in any case
depends on parameters that are difficult to measure in developing
countries.^{13} Thus, the central idea in applying it was
to classify the performance of the state capitals in relation to
the mortality rate and the mean length of stay, from fixed values
for the input variables. From this, the model described was
applied, in which the aim was to minimize inputs, i.e. to answer
the question of what proportional reduction in the inputs
(mortality rate and mean length of stay) it was possible to
achieve for a set of hospitals in one state capital while still
maintaining the observed disease profile and the mean amounts of
HAA reimbursements. The units (capitals) for which it was not
possible to reduce the variables were considered to be efficient
in comparison with the others, thus generating efficiency
scores.

The canonical correlation indicated that there was greater dependence between the variables "mean length of stay" (-0.724) and "neoplasia" (-0.656), which had the highest canonical coefficients among the variables analyzed (Table 2). Thus, it is inferred that, among the population studied, this was the group of diseases with the greatest impact on the patients' length of stay. This corroborates the hypothesis that neoplasias generally require greater length of stay, particularly regarding surgical conditions, and moreover, it shows that the same HAA procedure requires a longer stay if associated with a neoplasia group.

Using the scores generated by DEA, it could be seen that 16 state capitals were operating at less than 75% relative efficiency. The four cities identified as "100% efficiency" (Palmas, Macapá, Teresina and Goiânia) were not among the states with greatest per capita gross domestic product (GDP) or in which the country's major technological and educational centers are located. This indicates that, for the municipalities studied, significant performance gains are still possible with the existing supplies. This observation is reinforced by the independence between the classification scores and the variables "per capita expense on primary healthcare programs" and "HDI of the capitals". For example, the city of Macapá has one of the worst HDI and per capita expenses among the set of municipalities studied, but was classified as an "efficient unit". The HDI combines schooling, income and longevity data and is widely used as a quality-of-life indicator.

The capitals identified as having the worst performance had the most complex characteristics, and they included cities with a tradition of training healthcare human resources and other cities that, similar to the ones with the best performance, were distant from the country's main technological and educational centers.

One of the most important features of the methodology presented is it compares efficiencies while taking real functional conditions into consideration. Moreover, one original characteristic of the present study is the definition of weight limits for the variables, without the need for a decision-maker to intervene, since the restriction intervals were obtained from characteristics of the classification variables themselves (the inputs and outputs). The estimates for the mortality rate and mean length of stay may help health administrators by being a comparative reference point for clinical medicine indicators.

On the other hand, the work to improve these indicators does not dispense with identifying the intrinsic features of the units studied or other evaluations. For example, qualitative satisfaction surveys on the population attended may serve as parameters for demarcating the results. It is unlikely that any single reason for the relative positions of the state capitals will be identified, but the tool presented is a powerful and simple method for ranking performance, thereby opening the doors to more particular studies. Thus, the approach presented in this study is important and independent, and it provides managers with relevant information for wide-ranging evaluations of the system.

**REFERENCES**

1. Banker RD, Charnes A, Cooper WW. Some models for estimating technical and scale inefficiencies in Data Envelopment Analysis. *Management Science.*1984;30(9):1078-92. [ Links ]

2. Bouroche JM, Saporta G. Análise de Dados. Rio de Janeiro: Zahar Editores; 1982. [ Links ]

3. Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision-making units. *Eur J Oper Res*. 1978;2(6):429-444. [ Links ]

4. Chilingerian JA. Data Envelopment Analysis. Boston: Kluwer Academic Publishers; 1996. [ Links ]

5. Dexter F, O'Neill L. Data Envelopment Analysis to determine by how much hospitals can increase elective inpatient surgical workload for each specialty. *Anesth Analg.* 2004; 99(5):1492-500. [ Links ]

6. Färe R, Grosskopf S. New directions: efficiency and productivity. Boston: Kulwer Academy Publishers; 2004. [ Links ]

7. Felder S, Schmitt H. Data Envelopment Analysis based bonus payments. Theory and application to inpatient care in the German state of Saxony-Anhalt. *Eur J Health Econ.*2004;5(4):357-63. [ Links ]

8. Friedman L, Sinuany-Stern Z. Scaling units via the canonical correlation analysis in the DEA context. *European J Operational Res.* 1997;100(3):629–37. [ Links ]

9. Kirigia JM, Emrouznejad A, Sambo LG, Munguti N, Liambila W. Using Data Envelopment Analysis to measure the technical efficiency of public health centers in Kenya. *J Med Syst.* 2004;28(2):155-66. [ Links ]

10. Lins MPE, Meza LA. Análise Envoltória de Dados e Perspectivas de Integração no Ambiente de Apoio à Decisão. Rio de Janeiro: COPPE/UFRJ; 2000. [ Links ]

11. Marinho A. Estudo de eficiência em hospitais públicos e privados com a geração de rankings. *Rev Adm Publica.*1998;32(6):145-158. [ Links ]

12. Retzlaff-Roberts D, Chang CF, Rubin RM. Technical efficiency in the use of health care resources: a comparison of OECD countries. *Health Policy* 2004;69(1):55-72. [ Links ]

13. Valdmanis V, Kumanarayake L, Lertiendumrong J*.* Capacity in Thai public hospitals and the production of care for poor and nonpoor patients. *Health Serv Res.* 2004;39(6Pt 2):2117-34. [ Links ]

14. Viacava F, Almeida C, Caetano R Fausto M, Macinko J, Martins M, et al. Uma metodologia de avaliação do desempenho do sistema de saúde brasileiro. *Cienc Saude Coletiva.* 2004;9(3):711-724. [ Links ]

**
Correspondence:**

Renan M V R Almeida

Programa de Engenharia Biomédica Coppe

Universidade Federal do Rio de Janeiro

Caixa Postal 68510 Cidade Universitária

21941-972 Rio de Janeiro, RJ, Brasil

E-mail: renan@peb.ufrj.br

Received: 9/8/2005

Reviewed: 8/1/2006

Approved: 2/7/2007

1
Database of the Brazilian national health system [homepage on the Internet].
Brasília; 2005. Available at: http://tabnet.datasus.gov.br/
tabnet/tabnet.htm#AssistSaude [Accessed on March 3, 2005]

2 Banxia Frontier Analyst Professional.
Glasgow: Banxia Holdings Limited; 1998.

3 Lins MPE, Gonçalves AC, Gomes
EG, Silva ACM. Performance assessment of dental clinics through PC-oriented
Data Envelopment Analysis. Accessibility and quality of health services. In:
Proceedings of the 28th Meeting of the EURO Working Group Operational Research
Applied to Health Services; 2004; Frankfurt, Germany. Frankfurt: Peter Lang
Publishing Group; 2004. v. p. 95-109.

4
Fundação Nacional de Saúde. Consulta de pagamentos. Brasília;
2005. Available at: http://www.fns.saude.gov.br/consultafundoafundo.asp
[Accessed on March 3, 2005]

**Canonical correlation
analysis and CRS model**

**I - Canonical
correlation analysis (CCA)**

CCA was developed
by Hotelling in 1936. It studies linear relationships between two groups of
variables (*a* and *b*), and its fundamental concern is to find the
pair of linear combinations of *a* and *b* that has the maximum linear
correlation.^{3,11} From the scheme shown in the Table,
the linear combination of the variables *a* and *b* is defined as:

The coefficients
V_{i},i=1,....,m and U_{r},r=1,....,s must be such that the
square of the correlation between z and w, r^{2} (z,w), presents its
maximum value. It is assumed that the variables of the two groups are linearly
independent, i.e. the rank X_{mxn}=m and the rank Y_{sxn}=s.A_{1nxn}
are the orthogonal projectors of w_{nx1} and A_{2nxn} is the
orthogonal projector of z_{nx1}, i.e. A_{1} projects w in the
subspace Z and vice versa. The vector w must be collinear with the orthogonal
projection of z in W (the vector that makes a minimum angle with z).

This condition is expressed as:

A_{2}z
= rw

In which r = cos(z,w)
and A_{2} is the orthogonal projection operator in W.

Likewise:

A_{1} w
= rz

From this, the following can be deduced:

A_{1} A_{2}z
= r^{2} z and A_{2}A_{1}w = r^{2}w

In which l_{1}
= r^{2} = cos^{2}(z,w)

Consequently, **z**
and **w** are respectively eigenvectors of the operators A_{1}A_{2}
and A_{2}A_{1} that are associated with the greatest eigenvalue
l_{1} which is equal to its cosine squared (its squared correlation).

After appropriate
algebraic operations, and assuming that A_{2} can be inverted, the canonical
variables **z** and **w** can be written in the following form:

Likewise it can be deduced that:

The canonical variables
are the eigenvectors of A_{1}A_{2} (A_{2}A_{1})
, which are associated with the eigenvalues ranked in decreasing order. At each
stage, a pair of variables associated with the greatest eigenvalue () is generated.
The interest is in the canonical weights of the variables from the first stage
(greatest correlation), which are used in the proportions:

The values of these
weights indicate the importance of each variable in obtaining the maximum correlation
between the combinations, and can thus be utilized to generate restriction intervals
for the inputs and outputs in a DEA model. The matrixes A_{1} and A_{2}
and the canonical weights are obtained by:

That is, V(m x 1) and U(s x 1) are deduced from each other by linear transformation, such that D(n x n) is a diagonal weighting matrix of the variables.

**II - Constant
Returns to Scale (CRS) model**

In the case of
a unit with a single input-output pair, the efficiency of the unit can be defined
simply as the output/input ratio. In the case of several inputs and/or outputs,
the efficiency is the ratio between the weighted sum of the outputs and the
weighted sum of the inputs, and the following is a measurement of this efficiency:^{15}

(additional restrictions
for the weights, in accordance with the output and input levels of the j^{th}
DMU, respectively).

U_{r},
V_{i} >= 0, r=1, .... s and i=1, .... m are the weights (multipliers)
to be determined and e y_{rj}, x_{ij} >=0 are the outputs
and inputs known from the j^{th} DMU. The limits are obtained a *priori*,
by substituting the canonical weights of the inputs and outputs in the above
proportions, and they generate a value for each DMU. Consequently, there is
a set of *n* values for each variable, and the minimum and maximum for
each set define the limits and importance of each variable in the DEA, without
direct interference from a decision-maker.

The Figure illustrates the optimum input values that would turn an inefficient unit into an efficient one, according to this definition. In this particular case, points A, B, C and E correspond to inefficient units. Point D is an efficient unit, situated on the straight line that represents the efficient CRS frontier. The displacement of A to the efficient frontier (point P) implies the optimum input value that would make this unit efficient.