Criterion of unitary similarity for upper triangular matrices in general position

Abstract

Each square complex matrix is unitarily similar to an upper triangular matrix with diagonal entries in any prescribed order. Let A and B be upper triangular n-by-n matrices that (i) are not similar to direct sums of matrices of smaller sizes, or (ii) are in general position and have the same main diagonal. We prove that A and B are unitarily similar if and only if ||h(A_k)||=||h(B_k)|| for all complex polynomials h(x) and k=1, 2, . . , n, where A_k and B_k are the principal k-by-k submatrices of A and B, and ||M|| is the Frobenius norm of M.