Two-sided estimates for the transition densities of symmetric Markov processes dominated by stable-like processes in $C^{1,\eta}$ open sets

Abstract

In this paper, we study sharp Dirichlet heat kernel estimates for a large class of symmetric Markov processes in $C^{1,\eta}$ open sets. The processes are symmetric pure jump Markov processes with jumping intensity $\kappa(x,y) \psi_1 (|x-y|)^{-1} |x-y|^{-d-\alpha}$, where $\alpha \in (0,2)$. Here, $\psi_1$ is an increasing function on $[ 0, \infty )$, with $\psi_1(r)=1$ on $0<r \le 1$ and $c_1e^{c_2r^{\beta}} \le \psi_1(r) \le c_3 e^{c_4r^{\beta}}$ on $r>1$ for $\beta \in [0,\infty]$, and $ \kappa( x, y)$ is a symmetric function confined between two positive constants, with $|\kappa(x,y)-\kappa(x,x)|\leq c_5|x-y|^{\rho}$ for $|x-y|<1$ and $\rho>\alpha/2$. We establish two-sided estimates for the transition densities of such processes in $C^{1,\eta}$ open sets when $\eta \in (\alpha/2, 1]$. In particular, our result includes (relativistic) symmetric stable processes and finite-range stable processes in $C^{1,\eta}$ open sets when $\eta \in (\alpha/2, 1]$.