Modulation spaces of symbols for representations of nilpotent Lie groups

Abstract

We investigate continuity properties of operators obtained as values of the Weyl correspondence constructed by N.V. Pedersen (Invent. Math. 118 (1994), 1--36) for arbitrary irreducible representations of nilpotent Lie groups. To this end we introduce modulation spaces for such representations and establish some of their basic properties. The situation of square integrable representations is particularly important and in the special case of the Schr\"odinger representation of the Heisenberg group we recover the classical modulation spaces used in the time-frequency analysis.