Sufficiency-principled Transfer Learning via Model Averaging

Abstract

When the transferable set is unknowable, transfering informative knowledge as much as possible\textemdash a principle we refer to as \emph{sufficiency}, becomes crucial for enhancing transfer learning effectiveness. However, existing transfer learning methods not only overlook the sufficiency principle, but also rely on restrictive single-similarity assumptions (\eg individual or combinatorial similarity), leading to suboptimal performance. To address these limitations, we propose a sufficiency-principled transfer learning framework via unified model averaging algorithms, accommodating both individual and combinatorial similarities. Theoretically, we establish the asymptotic/high-probability optimality, enhanced convergence rate and asymptotic normality for multi-source linear regression models with a diverging number of parameters, achieving sufficiency, robustness to negative transfer, privacy protection and feasible statistical inference. Extensive simulations and an empirical data analysis of Beijing housing rental data demonstrate the promising superiority of our framework over conventional alternatives.