R-Impact: Reliability-Based Citation Impact Factor
Reliability-based citation impact (R-impact) factor considers the factors of both the citation impact, and long-lasting impact of published journals as measured by the cited half-life. The R-impact is simple to implement, yet a meaningful measure, which is defined as the cited half-life multiplied by the citation impact factor. This reliability-based citation impact factor (R-impact) gives an index of the life-span of the published journals, which can properly measure the effectiveness instead of just the short-term performance of the journals. The R-impact measures the life of published articles, as well as the marketability of the published articles as measured by the widely used short-term (two-year) citation impact factors. Introduction The recognized authority for evaluating journals, Journal Citation Reports, presents quantifiable statistics that provide a systematic, objective way to evaluate the world's leading journals, and their impact and influence in the global research community. Impact citations is a partial story of impact, and influence. Unfortunately, the research world has grasped firmly to one measure, the impact factor, as the de-facto measure of the effectiveness for a journal. This is clearly a misleading trend because the quality of a publication requires an effectiveness measure instead of just a performance measure of which the impact factor is only one, granted that it is an easy one to implement. A proper way to measure effectiveness must at least consider the long-term impact of a journal, among other factors. According to Garfield [1], [7], evaluation tools for journals have been described by the Expected Citation Rate (ECR), which provides a highly focused comparison of the impact of individual papers. Its inclusion, as part of the ISI ® Personal Citation Report, reduces invidious comparisons. Similarly, journal impact factors, and half-life measures provide more fair comparisons between fields of research with different rates of acceleration. The ECR has been used to compare the citation records of published items to the citation averages for similar items published in the same journal during the same database year. Deurenberg [2] uses ISI's impact factor, and a separately cited half-life to make decisions on journal selection, and weeding. R. Plomp [3] deals with the evaluation of a research group's performance. He states that it is likely better to measure the performance of a researcher instead of a group, as these measures tend to be applied. But he also states that long-term citation measures are better than short-term ones. Citation Impact Factor The impact factor is the most commonly used single factor to judge the quality of a journal. At this moment, it is almost exclusively used world-wide by universities, research institutions, and governmental agencies for judging the impact of a professional journal. This single factor has been used to determine promotion, funding opportunity, and many other decisions supporting the direction of science, yet has little to do with the quality of dissemination, and does not address the effectiveness of a journal. Impact factor is defined as the measure of the frequency with which the "average article" in a journal has been cited in a particular year [5]. The impact factor will help one to evaluate a journal's relative importance, especially when one compares it to others in the same field. It is calculated by dividing the number of current citations to articles published in the two previous years by the total number of articles published in the two previous years. Total cites is the total number of times that the journal has been cited by all journals included in the ISI ® database within the current product year. It is important to note that the impact factor is limited to a two year view. While that limit appears arbitrary, this type of metric does require some limit to be set as a standard. And because fields of science vary greatly, it is easy to argue how this metric will treat unfairly some areas of science, no matter what limit is set. Therefore, the two year period is a viable one to address, although new areas of science would be treated well with a short view such as the current two years, whereas developed fields of science would be treated well with a far longer view. It is clear that the impact factor as currently defined might serve as a viable measure for some subjects with timely issues, and perhaps has been a good performance measure for the medical and biological field, from where it originated. But there are established fields of science with very different ways of growing the state-of-the-art. Half-Life Cited half-life is defined as the number of publication years from the current year which account for 50% of current citations received [6]. This figure helps one to evaluate the age of the majority of cited articles published in a journal. Therefore, it is an obsolescence indicator, as interpreted by the radioactive physicists. Statisticians would refer to it as the median of a population. Half-life is a viable attribute of the longevity of the published articles, measured by the "life-span" of published papers. Citing half-life is the number of publication years from the current year that account for 50% of the current citations published by a journal in its article references [6]. This figure helps one to evaluate the age of the majority of articles referenced by a journal. Considering how these two measures interplay, having a high cited half-life is a very good accomplishment, especially given that we all pressure authors to cite recent work above all, driving down the citing half life. As we all drive down the citing half-life in our journals, we drive down the cited half-life for all others. In a reliable world, we either maximize the probability of the system's survival, or we maximize the residual life of the system. Therefore, half-life is the most creditable measure of the quality of published papers by a journal. Reliability-Based Citation Impact Factor: A Proposed Index Reliability-Based Citation Impact (R-impact) Factor considers the impact of the citation impact, and long-lasting impact of published journals as measured by the cited half-life. A simple yet meaningful measure is defined as the cited half-life multiplied by the citation impact factor. This reliability-based citation impact factor (R-impact) gives an index of the life-span of the published journals, which can properly measure the effectiveness instead of the short-term (two-year) performance of the journals. R-impact also considers how frequently the published papers are cited, known as the citation impact factors. Therefore the R-impact not only shows how marketable the published papers are, but gives a life to a journal which published the papers. This proposed R-impact index is easy to calculate from existing statistics, and in fact can be calculated from currently available ISI ® reports for most journals. See Figs. 1, and 2 for comparisons utilizing the available 2005 statistics of scientific publications we received from IEEE [8]; journals with a cited half-life greater than 10 years, such as IEEE Transactions on Reliability, do not have a cited half-life available in the report, so we will assume in the figure that their citation half-life is equal to 10 years.  Fig. 1. Comparison between our new Reliability-Based Citation Impact (R-impact) Factor versus the ISI-Defined Impact Factor.  Fig. 2. Comparison between the Cited Half-Life, and Impact Factor. For reference to Fig. 1, IEEE Transactions on Reliability scores a 7.15 for R-impact, ranks 56 among 176 measurable observations in the sample group of reported scientific publications with measurable results, and compares to an average R-impact value of 6.99 for the group. The figure shows obvious correlation between the measures, as would be expected. But see Fig. 2. The cited half life does not correlate strongly with impact factor, because they are measuring very different features of a journal. From the figure, we can see that there are no journals in the sample with a high impact factor, but a low citation half-life. But there are many with a long citation half-life, and smaller impact factors. By combining these two measures, we create a better measure of journal effectiveness without unfairly discounting those journals with high impact factor. Conclusions, and Options Impact Factor is not sufficient as a measure of effectiveness itself, and we believe we have stated a strong argument for why cited half-life should be considered as well. As a viable option to address this need, we present a new measure of journal effectiveness, termed Reliability-Based Citation Impact Factor (R-impact). We agree that there are other options to consider. While cumbersome, Plomp [3] might argue that we should measure the effectiveness of individual papers against others in their defined field, and report statistics on the papers published in a journal as that journal's effectiveness measure. But this approach requires rigid definitions of the various fields of science, presenting new problems. Another measure we would like to investigate is that of an impact factor with a timeframe equal to that of the cited half-life. However, the data necessary to calculate such a measure is not available to us at this time. And we have not fully considered the comparability of such a metric across journals with very different cited half-life measures. The commonly used citation impact factor measures the short term impact of a journal, and is applicable for the quick-evolving discoveries such as those of the biological type of journals. The proposed R-impact factor is meaningful, easy to implement, and gives due credits to all kinds of journals. The R-impact is equivalent to the spirit of Reliability, which can be used in a wide range of systems, from physical, to biological systems. References [1] Eugene Garfield, This essay was originally published in the Current Contents print editions September 12, 1994. [2] R. Deurenberg, Journal deselection in a medical university library by ranking periodicals based on multiple factors. Bull. Med. Libr. Assoc. 81(3):316-9, 1993. [3] R. Plomp, The highly cited papers of professors as an indicator of a research group's scientific performance. Scientometrics (29):377-93, 1994. [4] D J D. Price, Citation measures of hard science, soft science, technology, and nonscience. (Nelson C E & Pollack D K, eds.) Communication among scientists and engineers. New York: Columbia University Press, 1986, p. 155-79. [5] http://scientific.thomson.com/products/jcr/ [6] http://thomsonscientific.com/tutorials/jcrweb3/jcrtut05.htm [7] http://scientific.thomson.com/free/essays/citationanalysis/citationrates/ Expected citation rates, half-life, and impact ratios: comparing apples to apples in evaluation research [8] JCR 2005 Science Publications Data, available from ISI ®. |
||||