Popularization of Science

Popularization of Science

Evaluation research impacts: levels and indicators

Document Type : Original Article

Author
Ghasem Azadi
Assistant Professor of Information and Knowledge Science, National Research Institute for Science Policy, Tehran, Iran
10.22034/popsci.2024.424371.1306
Abstract
Purpose: Research evaluation is a set of different concepts and methods around quality and impact and is considered as a technical or operational activity that is designed to depict the objective elements of the research process. Understanding the value and effects of research can be tracked at different levels and with different indicators. This study lists and categorizes the quantifiable indicators that international reference bases and evaluation and rating systems have suggested.
Method: This research is practical in terms of its purpose, and it is considered a library in terms of the method of data collection. In order to extract measurable indicators of scientific impact, 29 databases and systems were examined.
Findings: The different levels of the science system that this study sought to identify their quantifiable indicators were categorized as follows: journals; articles; research areas; Books; researchers; Educational and research organizations and countries.
Conclusion: In order to evaluate the effects of research, there are various and different approaches and each of them has its own advantages and limitations. To better understand the role of scientometrics in impact evaluation, its position in the overall value chain of research should be seen, which starts with planning and funding; It continues with the selection and implementation of the project and ends with the research results. After use, results are achieved. Scientometrics is based on the quantitative analysis of scientific developments, mainly in the field of research results, trying to measure impact.
Keywords
"research impact evaluation"
"research effectiveness"
" research effect measurement"
"scientometric indicators"
Subjects

  • آزادی احمدآبادی، قاسم (1402). بررسی تأثیر انواع همکاری‌های علمی بر کیفیت آثار علمی ایران در حوزه زیست‌فناوری. مجله علم‌سنجی کاسپین،10 (2)، 64-53.

    آزادی احمدآبادی، قاسم (1402). ارزیابی تأثیرات بروندادهای علمی: مطالعه موردی حوزه زیست‌فناوری ایران. گزارش طرح پژوهشی. مرکز پژوهش‌ها  سیاست علمی کشور.

    آزادی احمدآبادی، قاسم؛ عبدی، ساجده؛ رمضانی، ابوذر. (1401). مطالعه تأثیرات علمی، اقتصادی و اجتماعی پژوهش‌های حوزه محیط‌زیست ایران. محیط‌زیست و توسعه فرابخشی،  7(78)،38-55.

    بابااکبری ساری، امیر؛ قهرمانی، محمد؛ فتحی واجارگاه، کوروش؛ موتمنی، علی‌رضا. (1400). ارائه الگوی ارزشیابی اثرات پژوهش‌های مدیریتی. پژوهش‌های مدیریت در ایران، 21(1)، 93-119.

    دانشگر، پرندیس؛ تقوی، فرشته؛ صبوری، علی‌اکبر؛ موسوی موحدی، علی اکبر. (1390). تولید دانش در علوم زیستی: پیشتاز در ایران و جهان. نشریه نشاء علم، 1(2)، 24.

    قاسمی، علی‌اصغر؛ امامی میبدی، راضیه. (1395). نقش و جایگاه مطالعات میان‌رشته‌ای در رشد و توسعه علوم انسانی در کشور. فصلنامه مطالعات میان‌رشته‌ای در علوم انسانی، 7(4)، 1-19.

    • Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature surveys. Journal of the American Society for information science and technology, 64(2), 217-233.
    • Boyack, K. W., Klavans, R., Sorensen, A. A., & Ioannidis, J. P. A. (2013). A list of highly influential biomedical researchers, 1996–2011. European Journal of Clinical Investigation, 43(12), 1339–1365. doi:10.1111/eci.12171.
    • Brown, S. (2009). A comparative review of research assessment regimes in five countries and the role of libraries in the research assessment process. OCLC Research.
    • Chi, P-S., Jeuris, W., Thijs, B. & Glänzel, W. (2015). Book Bibliometrics- A new perspective and challenge in indicator building based on the Book Citation Index. In A. A. Salah, Y. Tonta, A. A. A. Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of the 15th International Conference on Scientometrics and Informetrics (pp. 1161–1169). Istanbul, Turkey: Bogaziçi University Printhouse.
    • Cohen, G., Schroeder, J., Newson, R., King, L., Rychetnik, L., Milat, A. J. ... & Chapman, S. (2015). Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool. Health research policy and systems, 13(1), 1-12.
    • Douglas, H. (2014). Pure science and the problem of progress. Studies in History and Philosophy of Science Part A, 46, 55–63. Doi: 10.1016/j.shpsa.2014.02.001.
    • Finkel, A. (2014). Perspective: Powering up citations. Nature, 511(7510), S77. Doi: 10.1038/511S77a.
    • Giménez Toledo, E. (2020). Why books are important in the scholarly communication system in social sciences and humanities.
    • Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics, 101(2), 1419-1430.
    • Hazelkorn, E. (2011). Rankings and the reshaping of higher education. The battle for world-class excellence. New York, NY: Palgrave Macmillan.
    • Luhmann, N. (2012). Theory of society (Vol. 1). Stanford, CA: Stanford University Press.
    • Marjanovic, S., Hanney, S., & Wooding, S. (2009). A historical reflection on research evaluation studies, their recurrent themes and challenges. RAND Europe technical report. Available at: http://www.rand.org/pubs/technical_reports/2009/rand_tr789.pdf
    • Mingers, J. & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European Journal of Operational Research, 246, 1– 19. https:// doi.org/ 10.1016/ j.ejor.2015.04.002
    • Newson, R., King, L., Rychetnik, L., Bauman, A. E., Redman, S., Milat, A. J. ... & Chapman, S. (2015). A mixed methods study of the factors that influence whether intervention research has policy and practice impacts: perceptions of Australian researchers. BMJ open, 5(7), e008153.
    • Williams, K. (2020). Playing the fields: Theorizing research impact and its assessment. Research Evaluation, 29(2), 191-202.
    • Williams, K., & Lewis, J. M. (2021). Understanding, measuring, and encouraging public policy research impact. Australian Journal of Public Administration, 80(3), 554-564.
    • Wood, D.J., (1991). Corporate social performance revisited. The Academy of Management Review, 16(4), 691–718.
    • https://www.scopus.com/home.uri
    • https://www.webofscience.com
    • https://www.scival.com/landing
    • https://www.scimagojr.com/
    • https://incites.clarivate.com/
    • https://jcr.clarivate.com/
    • https://www.altmetric.com/explorer/login
    • https://plumanalytics.com/
    • https://books.google.com/
    • https://www.springernature.com/la/researchers/bookmetrix
    • https://scholar.google.com/
    • https://esi.clarivate.com/
    • http://www.eigenfactor.org/
    • https://icite.od.nih.gov/
    • https://www.topuniversities.com/
    • https://www.timeshighereducation.com/world-university-rankings
    • https://www.shanghairanking.com/rankings/
    • https://www.webometrics.info/en/world
    • https://www.scimagoir.com/rankings.php
    • https://www.leidenranking.com/
    • https://urapcenter.org/
    • https://wur.isc.ac/
    • https://www.timeshighereducation.com/world-university-rankings/by-subject
    • https://www.shanghairanking.com/rankings/gras/
    • https://urapcenter.org/
    • http://nturanking.csti.tw/
    • https://www.usnews.com/education
    • https://roundranking.com/ranking/rankings-overview.html