An Analogue of the Dedekind Eta Function for Hecke Groups $H(\sqrt{D})$

Abstract

Let $D\equiv 1\bmod{4}$ be a fundamental discriminant of a real quadratic field. We construct an analogue of the classical Dedekind eta function for the Hecke group $H(\sqrt{D})$. This gives rise to a new family of holomorphic modular functions for $H(\sqrt{D})$ which vanish at the cusp at $\infty$. We establish results on the asymptotic growth and sign patterns of the Fourier coefficients associated to these modular forms.