On the Markoff spectrum on the Hecke group of index six

Abstract

The discrete part of the Markoff spectrum on the Hecke group of index 6 was determined by A.~Schmidt. In this paper, we study its Markoff and Lagrange spectra after the smallest accumulation point $4/\sqrt3$. We show that both the Markoff and Lagrange spectra below $4/\sqrt{3} + \epsilon$ have positive Hausdorff dimension for any positive $\epsilon$. We also find maximal gaps and an isolated point in the spectra.