International Journal of Science, Engineering and TechnologyAuthors: Musale Devendra Kailas, Prof. J. M. Patade, Prof. M. C. Kolse, Prof. S. K. Said
Abstract: Large Language Models (LLMs) understand and generate human-like text, but general-purpose models often lack the vocabulary, knowledge and response style needed for specific domains such as healthcare, law, agriculture or customer support. Fine-tuning adapts a pre-trained model to a target task by continuing training on domain-specific data, which is far cheaper than training from scratch. This paper reviews fine-tuning methods for LLMs. It traces transfer learning from universal language model fine-tuning to text-to-text models and explains full fine-tuning, parameter- efficient fine-tuning with adapters, prefix and prompt tuning and low-rank adaptation, quantized fine-tuning, and instruction and preference tuning that align models with human expectations. Tools such as Hugging Face Transformers and PyTorch, and models such as BERT, GPT-style and LLaMA, are discussed. A five-stage proposed pipeline covering dataset collection, preprocessing, model selection, fine-tuning and evaluation is presented as a research design; no implementation results or accuracy claims are made. Research gaps, objectives, a comparison of fine-tuning methods and challenges including data quality, catastrophic forgetting, overfitting, compute cost, evaluation and safety are discussed. The review concludes that parameter-efficient methods make domain adaptation accessible to students and small organizations.
DOI: https://doi.org/10.5281/zenodo.24114144
