International Journal of Science, Engineering and TechnologyAuthors: Ritika Singh, Shiwangi Choudhary
Abstract: Machine translation for low-resource languages re- mains hindered by data scarcity and the prohibitive compu- tational cost of fully fine-tuning large multilingual models. To address this, we propose Language-Specific Fine-Tuning with LoRA (LSFTL), a parameter-efficient adaptation framework that enables high-quality translation for underserved language pairs using minimal bilingual data. LSFTL integrates lightweight, trainable Low-Rank Adaptation (LoRA) modules into a frozen pre-trained multilingual Transformer, with strategic selection of adaptation layers—focusing on attention projections and feed- forward networks—and coordinated encoder-decoder adaptation. This approach preserves the model’s extensive multilingual knowledge while specializing its behavior for a specific trans- lation direction. We evaluate LSFTL on multiple state-of-the- art models—including NLLB-200 and M2M-100—across several non-English-centric Asian language pairs (e.g., Hindi–Malay, Javanese–Tamil). Our results demonstrate that LSFTL achieves consistent and significant improvements, with gains of 1–3 COMET points and 5–7 BLEU points over zero-shot baselines, while attaining 97–99% of the performance of full fine-tuning. Crucially, LSFTL reduces trainable parameters by 99.2%, peak GPU memory usage by 61%, and training time by 74%, enabling billion-parameter model adaptation on a single consumer-grade GPU. LSFTL not only bridges the performance gap for low- resource languages but also offers a scalable and efficient pathway toward equitable machine translation.
DOI: http://doi.org/10.5281/zenodo.18373045
