Robust Bayesian Inference for Unnormalized Models with Mixed-Domain Data

Abstract

Many statistical models involve parameter-dependent normalizing constants that are computationally intractable, creating substantial obstacles to standard Bayesian inference. Although existing likelihood-based algorithms can often circumvent these constants, their uncertainty quantification may be poorly calibrated under model misspecification. To address these challenges, we propose SME-BETEL, a semiparametric Bayesian framework that combines score matching estimating equations with Bayesian exponentially tilted empirical likelihood. The resulting posterior avoids evaluation of normalizing constants and does not require learning-rate calibration. We establish consistency and asymptotic normality of the score matching estimator, and prove a Bernstein-von Mises theorem for the SME-BETEL posterior. These results show that SME-BETEL credible sets are asymptotically calibrated to the sampling variability of the score matching estimator, yielding valid frequentist coverage under model misspecification. We further develop a new score matching criterion for mixed-domain data, extending SME-BETEL to models whose observations combine components from different sample spaces. This construction enables robust Bayesian inference for mixed-domain doubly-intractable models, including preferential sampling models with an intractable spatial normalizing integral. Simulation studies show that SME-BETEL remains competitive under correct specification and substantially improves uncertainty quantification under misspecification. An ozone-monitoring application further demonstrates the utility of the mixed-domain construction for spatial preferential modeling.