Assessing the Reliability of Longitudinal Bibliometric Rankings under Incomplete Citation Histories
João Paulo Papa
Abstract
Longitudinal bibliometric analyses rely on annual citation histories, yet missing annual observations are frequently difficult to distinguish from true zero-citation years. We present a reproducible case-study framework for auditing citation-history gaps, tracing retrieval provenance, and quantifying the sensitivity of longitudinal indicators and rankings to explicit missing-data assumptions. The case study comprises 595 consolidated publications derived from 616 Google Scholar article records, and examines the completed calendar years 2024-2025. Three scenarios are compared: strictly observed annual counts (A), hypothetical zeros for internal gaps only (B), and hypothetical zeros for all missing years within a valid publication interval (C). The number of publications with computable growth and acceleration indicators increases from 196 (A) to 257 (B) and 490 (C); the corresponding counts for a revival indicator are 131, 152, and 177. Although the ordering of publications eligible under both compared scenarios is unchanged (Spearman and Kendall correlations equal to 1.0), complete-ranking positions and Top-K membership may change markedly. In particular, acceleration exhibits a Jaccard overlap of 0.563 at K=50 between A and C, while revival exhibits 0.667 at K=10. These findings demonstrate that unchanged indicator values and perfect correlations on a common subset do not guarantee stable rankings when eligibility changes. The zero assignments are hypothetical sensitivity assumptions, not evidence of provider semantics. The results are descriptive and based on a single researcher profile. A reusable Google Scholar Tracker software workflow supports the audit and sensitivity analysis; version 1.8.5 is publicly available on GitHub and archived on Zenodo (DOI: 10.5281/zenodo.24170285).
Create a lesson
Related papers
Do Network Mechanisms in Science Vary Across Research Topics? Contextual Relational Hyperevent Models for Large-Scale Bibliometric Data
Jürgen Lerner, Mark Wittek, Raphael Heiberger
Extracting Value from Grey Literature: processes and technologies for aggregating and analyzing the hidden big data treasure of organizations
G. Motta, R. Puccinelli, L. Reggiani et al.
Uncontrolled source titles: When Nature, Science, and PNAS meet Scopus
Weishu Liu, Ruifeng Zhang, Haifeng Wang
Theses by publication in Spain: output, authorship and integrity concerns
Álvaro Cabezas-Clavijo, Nazaret Barriga, Rafael Repiso
Where Do Highly Cited Scientists Stand? A Multi-Dimensional Analysis of Positional Distribution in the Top 2% Ranking
Yuehai Li, Zuguang Gu
Earlier citation practices are linked to signs of artificial intelligence use in scientific writing
Hongkan Chen, Yongyuan He, Qingshan Zhou et al.