INVESTIGACIÓN ORIGINAL ORIGINAL RESEARCH
Evaluación bibliométrica de la investigación sobre cáncer en Puerto Rico, 1903-2005
Ana P. OrtizI, II,1; William A. CaloIII; Carlos Suárez-BalseiroIV; Mariano Maura-SardoV; Erick SuárezII, III
ICancer Control and Population Sciences Program, University of Puerto Rico Comprehensive Cancer Center, San Juan, Puerto Rico
IIBiostatistics and Epidemiology Department, Graduate School of Public Health, Medical Sciences Campus, University of Puerto Rico, San Juan, Puerto Rico
IIIPuerto Rico Cancer Center, University of Puerto Rico, San Juan, Puerto Rico
IVObservatory of Informatics Related Studies, Graduate School of Information Sciences and Technologies, Río Piedras Campus, University of Puerto Rico, San Juan, Puerto Rico
VDeanship of Graduate Studies and Research, Río Piedras Campus, University of Puerto Rico, San Juan, Puerto Rico
OBJECTIVE:The analysis of scientific production in Puerto Rico is largely unexplored. The main aim of this study is to present the characteristics and trends of cancer publications in Puerto Rico's biomedical journals and their relationship with the island's cancer mortality.
METHODS:A PubMed and a hard-copy search were performed to retrieve cancer-related articles published in the Puerto Rico Health Sciences Journal and the Boletín de la Asociación Médica de Puerto Rico from 1903 to 2005. Bibliometric indicators studied included the number of authors and references by article, first author's institutional affiliation and country, document type, and language. The study type and tumor classification were also recorded. Cancer proportional mortality (M%) in Puerto Rico and the proportion of cancer-related articles (P%) published were also evaluated. The annual percent change (APC) was estimated to assess trends.
RESULTS:A total of 369 articles were retrieved. The institutions with the majority of publications were universities (39.6%), English was the predominant publication language (72.1%), and the principal document type was original papers (69.6%). Epidemiologic studies were the dominant study type (62.1%), and the most studied cancers were digestive (15.4%) and gynecologic (9.6%). Although the P% has increased since 1913 (APC = 1.2%), the M% increased at a faster pace (APC = 2.7%).
CONCLUSIONS: Although a growth in the number of cancer publications is observed in these journals, it does not parallel the increase in proportional mortality. A better understanding of the cancer publications in Puerto Rico is essential to establish priorities, define future areas of research, and develop cancer control strategies.
Key words: Bibliometric indicators, cancer, Puerto Rico.
OBJETIVO: El análisis de la producción científica de Puerto Rico está aún en gran parte pendiente. El objetivo principal de este estudio es presentar las características y las tendencias de las publicaciones sobre cáncer en las revistas biomédicas de Puerto Rico y su relación con la mortalidad por cáncer en la Isla.
MÉTODOS: Se realizó una búsqueda en PubMed y revistas impresas para localizar los artículos relacionados con cáncer publicados entre 1903 y 2005 en Puerto Rico Health Sciences Journal y el Boletín de la Asociación Médica de Puerto Rico. Entre los indicadores bibliométricos estudiados están el número de autores y referencias por artículo, la afiliación institucional y el país del primer autor, el tipo de documento y el idioma; se recogió también el tipo de estudio y de cáncer. Se evaluaron la mortalidad proporcional por cáncer (M%) en Puerto Rico y la proporción de artículos publicados relacionados con el cáncer (P%). Para evaluar las tendencias se estimó el cambio porcentual anual (APC).
RESULTADOS: Se localizaron 369 artículos. Predominaron los artículos originales (69,6%), de universidades (39,6%) y en inglés (72,1%). Los estudios epidemiológicos fueron los más frecuentes (62,1%) y los tipos de cáncer más estudiados fueron los del sistema digestivo (15,4%) y los ginecológicos (9,6%). Aunque el valor de P% ha aumentado desde 1913 (APC = 1,2%), el valor de M% aumentó más rápidamente (APC = 2,7%).
CONCLUSIONES: Aunque se observó un incremento en el número de publicaciones sobre cáncer en estas revistas, este no se correspondió con el aumento en la mortalidad proporcional por esta causa. Es imprescindible comprender mejor lo relacionado con la publicación científica sobre cáncer en Puerto Rico para establecer prioridades, definir áreas futuras de investigación y desarrollar estrategias para el control del cáncer.
Palabras clave: Indicadores bibliométricos, cáncer, Puerto Rico.
Bibliometry is the quantitative evaluation of scientific literature (1) defined as "the application of mathematics and statistical methods to books and other media of communication" (2). There are many ways in which bibliometric tools can be used (3). This discipline can be merely descriptive, but it can also be related to many kinds of indicators (4), such as disease burden (5). Bibliometric methods have many creative possibilities (6) and can help us map the intellectual growth of a discipline and help understand patterns of collaboration and funding (7). Bibliometry has become attractive to researchers and policy makers with its promise of some objective measure of output (8, 9). Even though bibliometry cannot be taken as an unequivocal measure of scientific quality, at higher aggregate levels, such as journals, bibliometric indicators give relevant information about research activity (10). Moreover, the application of bibliometric methods has been successful for the development of disease-specific public health initiatives in countries such as China and India (11, 12).
Even though bibliometry is a useful tool to evaluate scientific production (13), it has not been widely used in cancer research. Few bibliometric studies have been conducted with cancer research outputs (5, 14-20) and, to our knowledge, none has been performed on cancer-related articles published in Puerto Rican journals. Cancer is a public health problem in both industrialized and developing countries (21, 22) and remains a leading cause of morbidity and mortality worldwide (23). Already, the disease is the second cause of mortality in Puerto Rico (24), a country where the incidence rates of cancer have increased during the past three decades (25). Even though scientific activity has increased in Puerto Rico (26), the analysis of scientific production remains largely unexplored (27).
Although powerful industrialized countries are widely perceived as the main sources of scientific production, studies have shown that countries with a small geographic size can have high publication rates per inhabitant (4, 27). As a result of the increase in the burden of cancer in Puerto Rico and the lack of bibliometric studies, an evaluation of scientific literature published locally in the cancer field is highly warranted. This assessment would become a valuable resource to assist the scientific community, funding agencies, and governmental efforts to concentrate resources on strengthening the local cancer research capacity and on developing public health initiatives and cancer control strategies on the island. In addition, this type of information would be of particular relevance as it has been documented that investigators from Latin America and the Caribbean do not have the information resources necessary to identify research trends, make evaluative comparisons among nations, and track the results of public health programs (28). Furthermore, given that the two Puerto Rican biomedical journals are not included in most surveys conducted on scientific production in Latin America and the Caribbean (27) and only one of them has been included in the Institute for Scientific Information (ISI) Scientific Citation Index, alternative methods for evaluating the contribution of these two biomedical journals to the cancer field need to be assessed.
Thus, given the need of bibliometric studies in the cancer field in Puerto Rico, the objective of this study is to present the characteristics and trends of cancer-related publications in the two Puerto Rican biomedical journals from 1903 to 2005, using bibliometric indicators. In addition, we use two previously defined bibliometric indicators that allow for correlation between cancer-related publications and cancer mortality; these indicators permit an assessment of the relationship between biomedical research production and disease burden (5).
MATERIALS AND METHODS
The two currently existing peer-reviewed biomedical journals published in Puerto Rico, the Puerto Rico Health Sciences Journal (PRHSJ) and the Boletín de la Asociación Médica de Puerto Rico (BAMPR; known in English as the Puerto Rico Medical Association Bulletin), were analyzed. The PRHSJ has been the official journal of the University of Puerto Rico Medical Sciences Campus since 1982, while the BAMPR has been the official bulletin of the Puerto Rico Medical Association since 1903. Both journals publish scientific peer-reviewed articles in Spanish and English.
In accordance with the National Cancer Institute's cancer and tumor definitions (29), cancer publications for this study were defined as those that considered the following key terms: cancer, carcinoma, sarcoma, leukemia, lymphoma, multiple myeloma, malignant or cancerous tumor, and neoplasm.
Electronic and hard-copy search
A search was performed using the National Library of Medicine's search service PubMed (30) to retrieve cancer-related articles. Data retrieved included the complete list of available articles in PubMed until 2005; the PRHSJ is included since 1986 and the BAMPR since 1949. Key terms were searched for in Spanish and English using the medical subject heading (MeSH) tag and were limited to the PRHSJ and the BAMPR using the "search by journal" criteria. To include older articles not listed in the PubMed database and to corroborate the accuracy of the electronic retrieval, a manual hard-copy search was also done. Articles found as a result of the hard-copy search that were not found through the electronic search were looked for in PubMed, with the use of author names instead of key words.
An article was considered for inclusion in this study if it met the following criteria: (1) to be published in the PRHSJ or the BAMPR, and (2) to have at least one of the key terms defined previously in the title, abstract, or list of key words. For those articles that did not include an abstract or a list of key words, key terms were looked for only in the introduction or background section of the publication. Articles about benign or noncancerous tumors were excluded from this study, as they are not considered within the National Cancer Institute's cancer definition. All peer-reviewed papers including original, review, and special articles as well as other documents such as congress minutes, notes, letters, and editorials were eligible for inclusion. Journal supplements containing only meeting abstracts were excluded.
Our unit of analysis was cancer-related articles published in Puerto Rico during the period 1903 to 2005. Each article was classified by the source journal (PRHSJ or BAMPR) and the year of publication. The number of pages, authors, and references were collected for each article. The language of publication was classified as Spanish or English. The institution (University of Puerto Rico, other universities, non-university hospitals, government, non-university research centers, private industry, or not-for-profit organizations) and country (Puerto Rico, United States, or international) of institutional affiliation of the first author was also recorded. Manuscripts were also classified by their type as original contributions, congress minutes, notes, letters, editorials, and review and special articles. In addition, manuscripts were classified by study type: epidemiologic studies (case report, case series, incidence studies, cross-sectional studies, case-control studies, cohort studies, and clinical trials), basic sciences and translational studies (in vitro cancer cells and animal experiments), and studies in behavioral sciences. A fourth category was defined to include informative and educational articles. Tumor classification by body location or system, according to the National Cancer Institute (31), was recorded as AIDS-related, breast, digestive or gastrointestinal, endocrine, eye, genitourinary, germ cell, gynecologic, head and neck, hematologic or blood, musculoskeletal, neurologic or brain, respiratory, and skin. An additional category was added to classify articles that contained data on all cancer types as "all cancers combined."
An external validation of the methodology used to retrieve and classify the articles was done by an external reviewer not related to the study. A sample was randomly selected to conduct the validation process in both the BAMPR and the PRHSJ. The process included an electronic and a hard-copy search. The articles and the bibliometric indicators identified by the external reviewer were compared with those obtained in the original search. Similar results were obtained in both searches, supporting the assumption that the information originally retrieved was accurate.
To summarize and standardize the production of articles during the study period and between journals, the following indicators were computed:
- Annual average pages per article in year
- Annual average number of authors per article in year
- Annual average references per article in year , where i indicates the publication for a specific year.
The means and standard deviations (SD) of these indicators were computed overall and for each journal. In addition, contingency tables were used to describe the frequency distributions of the number of articles by journal and the following variables: institution and country of institutional affiliation of the first author, type of manuscript, language, study design, and cancer type. To assess the time trend in the bibliometric indicators and in the number of articles by different characteristics, the annual percent change (APC) was calculated by using the Poisson regression model (32). Finally, to relate the cancer research production to the cancer burden in Puerto Rico, we used the following parameters (5): (1) M%, representing the cancer proportional mortality ((number of cancer-related deaths)/(total number of deaths) ¥ 100) and (2) P%, representing the proportion of cancer-related articles ((number of cancer-related articles)/(total number of articles) ¥ 100). Information provided by the Annual Vital Statistics Reports of Puerto Rico (24, 33) was used to calculate M%. The total number of articles published was counted manually during the hard-copy search in order to define the denominator for P%. M% and P% data were restricted to 1913-1920 and 1923-2004, given that information on mortality was not available for the years 1903-1912, 1922-1923, and 2005. The statistical analysis was done with the computer software Stata/SE (Version 9.0, College Station, TX, USA).
A total of 39 articles published in the PRHSJ from 1986 to 2005 and 176 in the BAMPR from 1949 to 2005 were retrieved using PubMed and met the study's inclusion criteria. An additional 84 articles (76 from the BAMPR and 8 from the PRHSJ), not listed in PubMed during the same periods, were identified from both journals through the hard-copy search and 70 more from the periods not included in the PubMed database (64 from the BAMPR and 6 from the PRHSJ). Thus, 369 cancer-related articles were identified in 103 years of publication of these journals, with a mean of 3.6 articles per year.
As shown in Table 1, the BAMPR contributed with 316 (85.6%) articles during its 103 years in press, an average of 3.9 cancer-related articles per year, whereas 53 articles were identified in the PRHSJ (14.4%) from 1982 to 2005 (24 years in press), an average of 2.5 articles per year. An accelerated increase in the number of publications was observed during the complete study period (1903-2005), with an estimated APC of 2.3%. Original papers were the scientific document most often used to publish in Puerto Rico's biomedical journals (69.6%). A comparison of the journals during the period 1982-2005, when both journals were in existence, showed that the BAMPR had published more cancer-related articles than the PRHSJ. However, 86.8% of the articles in the PRHSJ were original papers, while 71.5% of the articles in the BAMPR were a similar type. Overall, the mean number of pages per article was 5.8 (SD 4.1); however, this mean was higher in the PRHSJ with 5.8 pages (SD 2.7) than in the BAMPR with 4.0 pages (SD 2.5) for the period 1982-2005. During this period, the mean number of authors per article was higher in the PRHSJ than in the BAMPR (4.7 (SD 3.9) and 2.7 (SD 1.6) authors, respectively); the same pattern was observed for the number of references per article (21.8 (SD 13.9) and 15.0 (SD 13.6) references, respectively).
Overall, the University of Puerto Rico was the institutional sector with the highest number of cancer publications (29.0%); hospitals ranked second (26.8%), followed by other universities (10.6%) and research centers (4.6%) (Table 1). Analysis during the comparative period (1982-2005) indicated that most of the first authors who published in the PRHSJ (60.4%) were affiliated with the University of Puerto Rico, while, for the BAMPR, a similar number of authors were affiliated with the University of Puerto Rico (30.9%) and hospitals (30.1%). In addition, even though most researchers (first authors) were affiliated with Puerto Rican institutions, a larger presence of researchers affiliated with institutions from the United States was observed in the PRHSJ (22.6%) than in the BAMPR (14.6%).
Overall (1903-2005), the use of English as the publication language was higher (72.1%) than the use of Spanish (27.9%), a pattern also observed in each specific journal (Table 1). Even though Spanish was the most common language for publication for the decades before 1950, English later became the most common language used for publication (data not shown).
Table 2 shows relevant methodologic characteristics of the evaluated publications. A comparison of both journals during the comparative period 1982-2005 showed some differences in the type of studies published. Most published articles in both journals were classified as epidemiologic studies (62.1%), followed by educational or informative publications (32.5%). Overall, the most studied cancer types were digestive (15.4%); gynecologic (9.6%), hematologic (9.1%), and genitourinary (8.6%). When analyzed specifically for the period 1982-2005, digestive, genitourinary, and skin cancers were more studied in the BAMPR, while breast, head and neck, and hematologic cancer studies were more frequently published in the PRHSJ.
The proportion of cancer publications (P%) with respect to the total publications in these two journals for the complete study period increased from 1.6% in the period 1913-1919 to 9.6% in the period 2000-2004 (Figure 1). Meanwhile, cancer proportional mortality (M%) in Puerto Rico also increased from 1.2% of all deaths in the same initial period to 16.7% of all deaths in the last period studied (2000-2004). Overall (1913-2003), the APCs showed a faster increase in P% than in M% (Figure 1).
This is the first study to assess cancer scientific production published in current Puerto Rican biomedical journals. We combined bibliometric and epidemiologic indicators to evaluate cancer scientific production in Puerto Rican journals, an approach that permitted an assessment of the relationship between biomedical cancer research production and cancer burden in this population. Our findings suggest a growing interest in cancer research in Puerto Rico, as shown by the increased number of cancer-related articles (APC = 2.3%). This result is consistent with other studies worldwide that have demonstrated an increase in cancer research publication in various countries, such as in Brazil (5) and Tunis (34), and by research topic, such as molecular cancer epidemiology (20). An increase was also observed for the proportion of cancer publications (P%) in these two local journals (APC = 1.2%), especially since the last mid-century. These increases match the epidemiologic transition of infectious diseases to chronic diseases described for the Puerto Rican population (35). Even if this transition has occurred, it is important to highlight that, although the proportion of cancer publications (P%) paralleled the cancer proportional mortality (M%) from the period 1913-1959, the burden of cancer in Puerto Rico in the last four decades has increased faster than the scientific production in cancer research published in these local journals (Figure 1). Notwithstanding, our proportion of cancer publications (P%) was higher than that reported for other countries, where cancer is one of the major scientific topics published (5, 34).
Given that, for most of the articles identified (68.8%), the country of institutional affiliation of the first author was Puerto Rico, these results also support the notion that, with the increasing burden of cancer in Puerto Rico observed during the past century, the local scientific community also increased its research and publication interests in the cancer field. This finding is consistent with research that shows that investigators tend to conduct research on topics of importance to their country or region (36). Our results highlight the need to further promote the publication of cancer research in these journals. This goal could be achieved by adding special cancer issues or supplements within both journals that could help to disseminate research results on a more international scale.
The fact that, for most of the articles retrieved, the country of institutional affiliation of the first author was identified as Puerto Rico also indicates that these journals are serving as a mechanism mainly for local researchers to publish their work and highlights an underutilization of these journals by international authors as a publishing mechanism. The impact factors of the PRHSJ and the BAMPR have not been determined yet, as the PRHSJ was included in the Science Citation Index Expanded in 2008 but the BAMPR is still outside the journals covered by ISI. Given the increasing interest of researchers in publishing their work in peer-reviewed journals with a high impact factor (37), the lack of visibility of these two Puerto Rican biomedical journals could be limiting their use, not only by international researchers but also by local researchers. Efforts should be made to include the BAMPR in the ISI Scientific Citation Index and to promote the recent indexing of the PRHSJ, which would increase their visibility worldwide and, thus, the desire of local and international researchers to publish in them (38, 39).
With respect to scientific collaboration, the observed average annual increase in the mean number of authors per article (APC = 2.7%) suggests an increase in collaboration among researchers. A previous bibliometric study in Puerto Rico also documented increases in collaboration, especially between local and international researchers (26). The same has been observed for Latin American countries (28). Scientific collaboration is rising in most disciplines due, among other factors, to the growing complexity of research projects, for which increasingly large numbers of researchers specializing in different areas are required (27). The cost-effectiveness of available resources is directly related to the size of research groups, with the quantity and quality of papers published increasing with the number of authors (40, 41). Also, greater collaboration can lead to higher-quality results because of the impact of scientists' collective experience and joint efforts (27). The rising tendency in the number of authors within scientific papers is a development identified in other studies as characteristic of research in the last decades in Puerto Rico (26) and internationally (42, 43), since scientific research today calls for more and better human and material resources (27).
The university was the institutional sector with the highest number of cancer-related articles published and, specifically, the University of Puerto Rico was the most active center in this regard. A similar pattern was observed for scientific production in science and technology in Puerto Rico during the period 1990-1998 (27). This finding is in keeping with the institutional goal of making the University of Puerto Rico the leading science and technology institution and its researchers more active and competitive at both local and international levels (44, 45). It should be noted in this connection that the executive and legislative branches of the government in Puerto Rico perceive scientific research as an activity characteristic of the university; therefore, the infrastructure of governmental agencies has not been ideally designed for scientific research (46).
English was the predominant language used for publication. The reasons for the common use of the English language in Puerto Rican scientific literature have to do with the political, legal, and economic relations with the United States that characterize scientific collaboration, availability of funds for research, and implementation of research projects in Puerto Rican institutions (27). Another factor is the broad acceptance of English as the international language for pure, experimental, and technological sciences (47).
Original articles were the publication type most commonly found in local journals. This result coincides with trends observed in previous studies conducted in science and technology in Puerto Rico (27, 40) and internationally (48). Meanwhile, most publications identified were epidemiologic studies-more specifically, epidemiologic descriptive studies (case-report, case-series, incidence, and cross-sectional studies). Even though these descriptive studies do not offer the strongest evidence to assess causality in epidemiologic studies, they provide important information on the etiology of disease and help identify potential risk factors that could be better evaluated for causality in analytical epidemiologic studies (cohort studies, case-control studies, and clinical trials) (49, 50). In fact, findings show a limited number of publications based on analytical studies and basic and translational studies. Some factors that could discourage investigators from conducting these types of studies include the fact that they are more expensive, take more time to conduct, and often require larger sample sizes than descriptive studies (51). Another explanation for the limited number of analytical studies published in these journals could be that authors might prefer to publish these types of studies in foreign journals with a higher impact factor (37). Even with the challenges of these types of studies, they should be further promoted in the cancer field, as they constitute the strongest methodologies to further evaluate causality between risk factors and disease development. With respect to the large quantity of educative/informative articles identified, even though they are important for educating the reader on cancer risks and prevention, they do not contribute substantially toward advancing the existing scientific knowledge in the field, as they do not involve a formal scientific research process.
Digestive, breast, gynecologic, and genitourinary cancers-the cancer types most studied in local publications-coincide with the cancer types with a historically high incidence and mortality in Puerto Rico, as reported by the Puerto Rico Central Cancer Registry (52). Recent data from the Puerto Rican Cancer Registry support the fact that these cancer sites continue to be the most common cancers on the island. In the year 2003, digestive and genitourinary cancers accounted for the majority of incident cases of cancer in Puerto Rican males; among women, most incident cases of cancer were attributed to breast, digestive, genitourinary, and gynecologic cancers (53). These data support the idea that, given the continued high burden of cancer attributed to these body locations in Puerto Rico, they should continue to be a primary focus of local cancer scientific production in the future.
Our results provide valuable information that could be obtained only by this unique analysis of hard-copy retrieval, given that publications in the studied journals are not totally included in PubMed or indexed in the ISI system. Although PubMed is the most widely used biomedical system (54, 55), indexing more than 4 600 journals worldwide (56), not all of Puerto Rico's research publications are indexed in it. Approximately 17% of the manually retrieved articles were not available in PubMed, as they had been published before the date when their journals of precedence were incorporated in that electronic database. Also, 25% of the manually retrieved articles were not available in PubMed, although they were published within the period when both journals were included in the electronic database. The same has been documented for other countries, such as Australia and Brazil (5, 57, 58). Prior studies comparing the comprehensiveness and relevance of PubMed and other databases have demonstrated that a single search engine does not capture all the pertinent and available articles retrieved (59). To improve our search, MeSH terms, the main key words used within the PubMed database, were used in our methodology to retrieve cancer-related articles. Given that the specificity of PubMed retrievals has been shown to increase when MeSH words are used (57), we expect our electronic retrieval to have been adequate.
The previous findings support the relevance of performing a hard-copy search to identify all the cancer-related articles published by both journals in order to better understand the number and type of cancer-related articles published in Puerto Rican journals. Given that many journals from Latin American countries are not totally included in PubMed or indexed in the ISI databases (60), a well-established hard-copy retrieval method is an excellent way to conduct descriptive bibliometric studies. Since 1998, the Scientific Electronic Library Online (SciELO) has provided universal visibility and accessibility of scientific literature from several developing countries, particularly from Latin America and the Caribbean (61). Nonetheless, Puerto Rico is not included as a country in the SciELO database.
A complete understanding of citing, publishing, and collaboration patterns in Puerto Rico is critical to researchers, policy makers, and heath care professionals in order to make informed decisions about research priorities (62) and public health interventions and to guide Puerto Rico's Cancer Control Plan (63). Although our study provides important information on publication patterns in local Puerto Rican biomedical journals, a complete analysis of cancer publications published in international journals by researchers affiliated with Puerto Rican institutions is being done by our research team. Preliminary results from this work have shown that a larger number of cancer articles were published in international journals than in local journals during the period 1992-2004 (63).2
The lack of standardization in the structure of the publications retrieved in the BAMPR before establishment of the Uniform Requirements for Manuscripts Submitted to Biomedical Journals in 1979 (64) and the lack of key words in several publications from both journals were a difficulty encountered in the hard-copy retrieval that might have affected our ability to identify all cancer-related publications through the hard-copy search. Nonetheless, the extraction of key words from the introduction and background sections of the papers reduced the possibility of that happening. A journal's system of key words ideally should include items such as disease and its body location, which would facilitate retrieval methods (4).
In conclusion, our study showed that most cancer-related publications in the PRHSJ and the BAMPR, two Puerto Rican biomedical journals, from 1903 to 2005 were original articles, were written in English, and were from authors affiliated with academic institutions. In addition, epidemiologic studies were the dominant study type and the most studied cancers were digestive and gynecologic. With regard to publication trends, the growth observed in cancer publications in these journals for the complete study period did not parallel the increase in the cancer proportional mortality in Puerto Rico. To fully understand the total spectrum of cancer research in Puerto Rico, it is essential that future bibliometric studies also evaluate other local scientific materials such as monographs, books, and dissertation theses as well as articles published in journals outside Puerto Rico by authors affiliated with Puerto Rican institutions. This will permit a comprehensive assessment of the current research taking place with regard to the total amount of cancer scientific production published in Puerto Rico. Future studies should also include citation analyses for authors affiliated with Puerto Rican institutions as an important consideration for evaluative purposes of research quality. Finally, the characteristics and publication rate of cancer scientific production in Puerto Rico should be compared with that of other countries to further understand our level of development in the area of cancer research.
Acknowledgments. The authors acknowledge Eduardo Hernández, MPH (Puerto Rico Cancer Center), and Laura Bretaña, MA (Puerto Rico Cancer Center), for their support in the manuscript's preparation; Guillermo Tortolero-Luna (University of Puerto Rico Comprehensive Cancer Center) and Carmen Santos, MSL (University of Puerto Rico Medical Sciences Campus Library), and the Medical Association of Puerto Rico for facilitating access to all the BAMPR journals. This project was supported by National Institutes of Health (NIH) Grant U54CA96297 from the Puerto Rico Cancer Center/M.D. Anderson Cancer Center Partnership, Research Centers in Minority Institutions Grant G12RR03051 from the University of Puerto Rico, and NIH Grant 5P20RR011126 from the National Center for Research Resources.
1. Luukkonen T. Bibliometrics and evaluation of research performance. Ann Med. 1990;22(3): 145-50.
2. Pitchard A. Statistical bibliography or bibliometrics. J Doc. 1969;25(4):348-9.
3. Glover SW, Bowen SL. Bibliometric analysis of research published in Tropical Medicine and International Health 1996-2003. Trop Med Int Health. 2004;9(12):1327-30.
4 Cimmino MA, Maio T, Ugolini D, Borasi F, Mela GS. Trends in otolaryngology research during the period 1995-2000: a bibliometric approach. Otolaryngol Head Neck Surg. 2005; 132(2):295-302.
5. Rodrigues PS, Fonseca L, Chaimovich H. Mapping cancer, cardiovascular and malaria research in Brazil. Braz J Med Biol Res. 2000; 33(8):853-67.
6. Wallin JA. Bibliometric methods: pitfalls and possibilities. Basic Clin Pharmacol Toxicol. 2005;97(5):261-75.
7. Traynor M, Rafferty AM. Bibliometrics and a culture of measurement. J Adv Nurs. 2001; 36(2):167-8.
8. Smith K, Marinova D. Use of bibliometric modelling for policy making. Math Comput Simul. 2005;69(1-2):177-87.
9. Montes GS. Distribution of financial resources according to the productivity (determined by bibliometry) in the Medical Research Laboratories of the Hospital das Clinicas da Faculdade de Medicina de Universidad de Sao Paulo (Brasil). Rev Med Chil. 2000;128(4):431-6.
10. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):497-502.
11. Arunachalan S, Gunasekaran S. Tuberculosis research in India and China: from bibliometrics to research policy. Curr Sci. 2002;82(8): 933-47.
12. Arunachalan S, Gunasekaran S. Diabetes research in India and China today: from literature-based mapping to health-care policy. Curr Sci. 2002;82(9):1086-97.
13. Roy D, Hughes JP, Jones AS, Fenton JE. Citation analysis of otorhinolaryngology journals. J Laryngol Otol. 2002;116(5):363-6.
14. Grossi F, Belvedere O, Rosso R. Geography of clinical cancer research publications from 1995 to 1999. Eur J Cancer. 2003;39(1):106-11.
15. Parodi S, Parodi A, Lombardo C, Santi L. Cancer research in the European community and other non-EC countries. Tumori. 1993;79(1):9-15.
16. Mela GS, Cimmino MA, Ugolini D. Impact assessment of oncology research in the European Union. Eur J Cancer. 1999;35(8): 1182-6.
17. Ugolini D, Casilli C, Mela GS. Assessing oncological productivity: is one method sufficient? Eur J Cancer. 2002;38(8):1121-5.
18. Mahoney MC, Michalek AM. A bibliometric analysis of cancer among American Indians & Alaska Natives, 1966-1993. Alaska Med. 1995;37(2):59-62, 77.
19. Lichtman MA, Oakes D. The productivity and impact of the Leukemia & Lymphoma Society Scholar Program: the apparent positive effect of peer review. Blood Cells Mol Dis. 2001;27(6):1020-7.
20. Ugolini D, Puntoni R, Perera FP, Schulte PA, Bonassi S. A bibliometric analysis of scientific production in cancer molecular epidemiology. Carcinogenesis. 2007;28(8):1774-9.
21. Jones LA, Chilton JA, Hajek RA, Iammarino NK, Laufman L. Between and within: international perspectives on cancer and health disparities. J Clin Oncol. 2006;24(14):2204-8.
22. Kamangar F, Dores GM, Anderson WF. Patterns of cancer incidence, mortality, and prevalence across five continents: defining priorities to reduce cancer disparities in different geographic regions of the world. J Clin Oncol. 2006;24(14):2137-50.
23. Kanavos P. The rising burden of cancer in the developing world. Ann Oncol. 2006;17(Suppl 8):viii, 15-23.
24. Departamento de Salud de Puerto Rico. Reporte de Estadísticas Vitales de Puerto Rico, 2003. San Juan, Puerto Rico: Departamento de Salud; 2005.
25. Figueroa NR, De la Torre T, Ortiz KJ, Pérez J, Torres M. All sites cancer stat fact sheet. San Juan, Puerto Rico: Puerto Rico Central Cancer Registry. Available from: http://www.salud. gov.pr/RCancer/Reports/Documents/Hojas%20informativas/All%20Sites.pdf. Accessed 20 January 2009.
26. Suárez-Balseiro C, Sanz-Casado E, Ortiz-Rivera L. Patterns of international scientific co-operation in Puerto Rico. Scientometrics. 2006;67(3):335-50.
27. Ortiz-Rivera L, Sanz-Casado E, Suarez-Balseiro CA. Scientific production in Puerto Rico in science and technology during the period 1990 to 1998. Scientometrics. 2000;49(3): 403-18.
28. Macias-Chalupa C. Hacia un modelo de comunicación en salud publica en América Latina y el Caribe. Rev Panam Salud Publica. 2005;18(6):427-38.
29. National Cancer Institute. Dictionary of cancer terms. Bethesda, MD: National Institutes of Health. Available from: http://www.cancer.gov/dictionary. Accessed 17 April 2006.
30. National Library of Medicine. PubMed. Bethesda, MD: National Institutes of Health. Available from: http://www.ncbi.nlm.nih. gov. Accessed 19 April 2006.
31. National Cancer Institute. A to Z list of cancers. Bethesda, MD: National Institutes of Health. Available from: http://www.cancer. gov/cancertopics/cancersbodylocation. Accessed 19 April 2006.
32. Hardin J, Hilbe J. Generalized linear models and extensions. College Station, TX: STATA Press; 2001.
33. Departamento de Salud de Puerto Rico. Reporte de estadísticas vitales de Puerto Rico, 1970. San Juan, Puerto Rico: Departamento de Salud; 1972.
34. Ben Abdelaziz A, Harrabi I, Aouf S, Gaha R, Ghannem H. Typology of Tunisian medical research indexed in MEDLINE from 1965 to 1999. Tunis Med. 2002;80(9):548-55.
35. Rigau-Pérez JG. Health in Puerto Rico in the 20th century. P R Health Sci J. 2000;19(4): 357-68.
36. Hofman K, Ryce A, Prudhomme W, Kotzin S. Reporting of non-communicable disease research in low- and middle-income countries: a pilot bibliometric analysis. J Med Libr Assoc. 2006;94(4):425-20.
37. Bordons M, Zulueta MA. Evaluation of scientific activity though bibliometric indices. Rev Esp Cardiol. 1999;52(10):790-800.
38. Garfield E. How ISI selects journals for coverage: quantitative and qualitative considerations. Current Contents. 1990;22:5-13.
39. Ruiz-Pérez R, Delgado López-Cózar E, Jiménez-Contreras E. The Institute for Scientific Information journal selection criteria. Its application to Archivos de la Sociedad Española de Oftalmología. Arch Soc Esp Oftalmol. 2006;81(5): 245-68.
40. Sanz E. Bibliometric study of the scientific production of the University of Puerto Rico, Río Piedras Campus, Department of Chemistry from 1989 to 1994. San Juan, Puerto Rico: University of Puerto Rico; 1995.
41. Gordon MD. A critical assessment of inferred relations between multiple authorship, scientific collaboration, the production of papers and their acceptance for publication. Scientometrics. 1980;2(3):193-201.
42. Glanzel W, De Lange C. Modelling and measuring multilateral co-authorship in international scientific collaboration. Part II. A comparative study on the extent and change of international scientific collaboration links. Scientometrics. 1997;40(3):605-26.
43. Glanzel W, Schubert A, Czerwon HJ. A bibliometric analysis of international scientific cooperation of the European Union (1985- 1995). Scientometrics. 1999;45(2):185-202.
44. Maldonado N. Education on the road to excellence, all one system. Arlington, VA: The National Science Foundation Available from: http://www.nsf.gov/nsb/meetings/1999/oct/oct7.htm. Accessed 25 October 2006.
45. Universidad de Puerto Rico. X para la década, agenda para la planificación de en la Universidad de Puerto Rico: 2006-2016. San Juan, Puerto Rico: Universidad de Puerto Rico; 2005.
46. Fernández JR. Política publica para la ciencia en Puerto Rico. Ciencia y política en Puerto Rico. San Juan, Puerto Rico: Ateneo Puertorriqueño; 1990. Pp. 37-44.
47. Gomez I, Bordons M. Limitations in the use of bibliometric indicators for the scientific evaluation. Polit Cientifica. 1996;46:21-6.
48. Ming-yueh T. Bibliometric analysis of the literature of randomized controlled trials. J Med Libr Assoc. 2005;93(4):450-8.
49. Hennekens C, Buring JE. Epidemiology in medicine. 1st ed. London: Lippincott Williams and Wilkins; 1987. Pp. 16-28.
50. Szklo M, Nieto FJ. Epidemiology beyond the basics. 2nd ed. Sundbury, MA: Jones and Bartlett Publishers; 2007. Pp. 3-4.
51. Merrill RM, Timmreck TC. Introduction to epidemiology. 4th ed. Sundbury, MA; Jones and Bartlett Publishers; 2006. Pp. 165-98.
52. Departamento de Salud de Puerto Rico. Reporte del Registro Central de Cáncer, 1991. San Juan, Puerto Rico: Departamento de Salud; 1993.
53. Departamento de Salud de Puerto Rico. Reporte preliminar del Registro Central de Cáncer, 2003. San Juan, Puerto Rico: Departamento de Salud; 2007.
54. Pestaña A. Suitability of MEDLINE for the study of the Spanish scientific production in biomedicine and medical sciences. A comparative appraisal with the Science Citation Index. Med Clin (Barc). 1997;109(13):506-11.
55. Pritchard SJ, Weightman AL. MEDLINE in the UK: pioneering the past, present and future. Health Info Libr J. 2005;22(Suppl 1):38-44.
56. Mendis K, McLean R. Increased expenditure on Australian health and medical research and changes in number of publications determined using PubMed. Med J Aust. 2006;185 (3):155-8.
57. Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled clinical trials using PubMed. Int J Epidemiol. 2002;31(1): 150-3.
58. Mendis K. Health informatics research in Australia: retrospective analysis using PubMed. Inform Prim Care. 2007;15(1):17-23.
59. Wilkins T, Gillies RA, Davies K. EMBASE versus MEDLINE for family medicine searches: can MEDLINE searches find the forest or a tree? Can Fam Physician. 2005;51:848-9.
60. Pellegrini A, Goldbaum M, Silvi J. Producción de artículos científicos sobre salud en seis países de América Latina, 1973 a 1992. Rev Panam Salud Publica. 1997;1(1):23-34.
61. Scientific Electronic Library Online. Availa-ble from: http://www.scielo.org. Accessed 18 June 2008.
62. Burright MA, Hahn TB, Antonisse MJ. Understanding information use in a multidisciplinary field: a local citation analysis of neuroscience research. Coll Res Libr. 2005;66(3): 198-210.
63. Puerto Rico Cancer Control Coalition. Puerto Rico Comprehensive Cancer Control Plan 2008-2012. A blue-print for cancer control in Puerto Rico. San Juan, Puerto Rico: Puerto Rico Cancer Control Program; 2008.
64. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Available from: http://www.icmje.org. Accessed 9 July 2008.
Manuscript received on 18 December 2007.
Revised version accepted for publication on 10 July 2008.
1 Send correspondence and reprint requests to: Ana Patricia Ortiz, Biostatistics and Epidemiology Department, Graduate School of Public Health, Medical Sciences Campus, University of Puerto Rico, PO Box 365067, San Juan, Puerto Rico 00936-5067; telephone: (787) 758-2525, extension 1427; fax: (787) 764-5831; e-mail: email@example.com
2 Ortiz AP, Calo WA, Suárez CA, Suárez E, Iribarren I, Sanz-Casado E. Basic characteristics of cancer-related research in Puerto Rico; an approach to local and international journals. International Society for Scientometrics and Informetrics; 25-27 June 2007; Madrid, Spain.