CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation
Nishant Kambhatla, Logan Born, Anoop Sarkar
Abstract
We propose a novel data-augmentation technique for neural machine translation based on ROT-k ciphertexts. ROT-k is a simple letter substitution cipher that replaces a letter in the plaintext with the kth letter after it in the alphabet. We first generate multiple ROT-k ciphertexts using different values of k for the plaintext which is the source side of the parallel data. We then leverage this enciphered training data along with the original parallel data via multi-source training to improve neural machine translation. Our method, CipherDAug, uses a co-regularization-inspired training procedure, requires no external data sources other than the original training data, and uses a standard Transformer to outperform strong data augmentation techniques on several datasets by a significant margin. This technique combines easily with existing approaches to data augmentation, and yields particularly strong results in low-resource settings.
Create a lesson
Related papers
Mechanistic Interpretability Needs a Logic of Inference
Zhimin Hu, Chen Jiang, Hanbo Xie et al.
Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation
Evelyn Y. Garland, Carola F. Berger
When does text encoding matter for language modeling?
Eugene Jang, Catherine Arnett, Terra Blevins
Blurring the Block Boundaries: Overlapping Block Denoising for Efficient Diffusion Language Models
Amir Mann, Jonathan Fhima, Shaked Brody et al.
ARE-bench: Benchmarking Agents' Research Ability in Real-World Researcher Sessions and Workspaces
Jiahang Lin, Wanying Ren, Wanghan Xu et al.
Point-in-Time Encoders for Earnings Calls
Alessio Brini