Improved deterministic integer factorization by quadratic-image sieving

Abstract

Let $N=pq$ be a semiprime and let $\ell\nmid N$ be an odd prime. For $a\ne0$ modulo $\ell$, put $H_a(N;\ell)=\{ax+Nx^{-1}:x\in\F_\ell^*\}$. For a primitive root $γ$, the size of $H_γ(N;\ell)\cup H_{γ^2}(N;\ell)$ is determined by the Frobenius trace of $Y^2=(X^2-1)(X^2-γ)$. We study the resulting trace-selection problem through finite certificates, a congruence modulo $8$, and a GRH-conditional positive-density family from the CM specialization $γ=2$. Independently of this conditional result, we obtain a deterministic factorization algorithm from the residue-known coefficient lattice of Harvey and Hittmeir (\emph{Math. Comp.}, 2022). The actual babystep index is a nondegenerate quadratic image modulo each selected prime divisor of the auxiliary modulus. After a computable translation, every such image avoids a fixed set of $(\ell-χ(-1))/4$ residues. This forbidden set is optimal among translated sets that must handle both square classes of the leading coefficient. Gauss reduction, a nearby lattice-vector selection, arbitrary-point multipoint evaluation, and the unrestricted large-order algorithm of Harvey and Hittmeir (arXiv, 2026) give the bit-complexity bound $N^{1/5}\exp(-(\log(4/3)/50+o(1))\log N/\log\log N)$. Its secondary factor is asymptotically smaller than every fixed negative power of $\log N$, improving the general bound of Harvey and Hittmeir (\emph{Math. Comp.}, 2022) and the balanced-semiprime bound of Gao et al.\ (arXiv, 2025).