Adaptation of a Large Language Model to a Specific Domain using Instruction Following
Overview
This pipeline provides code and a set of instructions on how to adapt a large language model to a specific domain, e.g., a specific industry or scientific area. The prerequisites are a sufficient amount of domain-specific data and domain know-how. The pipeline first helps users in preparing data for instruction following based on the domain data and provides code and know-how for expanding the initial small sample of domain-specific instructions into a larger training dataset. Next, it provides the know-how and code for fine-tuning the model with the created instruction-tuning dataset. Finally, it provides code for efficient inference using the adapted model. A worked example of adaptation is provided. The pipeline uses the Slovene LLM GaMS as well as other state-of-the-art open large language models and libraries that support instruction set expansion, LLM fine-tuning with LoRA and QLoRA, and efficient inference with vLLM.
Outcome
Documentation for domain adaptation and code for domain adaptation, enabling users to adapt LLMs to their specific domain.
Service Card
| Category | AI development and deployment |
| Domain | Natural Language Processing, Domain Adaptation |
| Vertical | Cross-domain |
| Target users | Industry, Large Enterprises, Scientific Institutions, Public Institutions |
| Service type | Pipeline |
| Outputs | Domain-adapted model for training; generated text for inference. |
| Requirements | Requires several NVIDIA GPU nodes. |
| Limitations | |
| Availability | Q3/2027 |
| Licensing | MIT |
| Visual content | |
| Visual caption |