LLM Fine-Tuning
Decide when fine-tuning is warranted, then do it properly: dataset construction, LoRA and QLoRA, hyperparameters, evaluation, and serving the result.
Course Overview
Most teams that want to fine-tune a model should not. Prompt engineering, better retrieval, or a stronger base model solves the problem more cheaply and without creating an artifact that must be maintained, versioned, and re-evaluated every time the base model updates. This course opens by making that case honestly, with a decision framework for when fine-tuning genuinely pays: consistent output structure, domain vocabulary the base model handles poorly, latency or cost targets that require a smaller model, or behavior that resists prompting.
Once the case is established, the work is mostly data. Fine-tuning quality is dominated by dataset construction, and a few thousand carefully curated examples routinely beat a hundred thousand scraped ones. You will build a dataset, handle formatting and tokenization, deliberately introduce and then detect common problems such as label leakage and distribution mismatch, and learn why a clean held-out split matters more here than almost anywhere else.
The technical core is parameter-efficient fine-tuning. You will run LoRA and QLoRA on consumer-grade GPUs, work through rank and alpha selection, learning-rate schedules, and the memory arithmetic that determines what fits. The course closes on evaluation against the base model and on serving: adapter merging, multi-adapter hosting, quantization for inference, and the regression testing that catches catastrophic forgetting before users do.
Duration: 3 days|Delivery: onsite, virtual, or hybrid
Prerequisites
- Strong Python and hands-on experience with PyTorch
- Working understanding of transformer architecture and attention
- Experience training or evaluating machine learning models
- Familiarity with GPU compute and CUDA basics
Who Should Attend
- ML engineers evaluating fine-tuning against prompting and RAG for a specific requirement
- AI engineers who need a smaller, cheaper model to hit a latency or cost target
- Research engineers adapting open-weight models to specialized domain language
- Technical leads who must budget the GPU and maintenance cost of a fine-tuned model
Course Outline
- 1When to fine-tune and when not to: a decision framework against prompting, RAG, and model upgrades
- 2Fine-tuning taxonomy: continued pretraining, supervised fine-tuning, and preference optimization
- 3Dataset construction: sourcing, curation, deduplication, and target dataset size
- 4Data quality failure modes: label leakage, distribution mismatch, and near-duplicate contamination
- 5Formatting, chat templates, tokenization, and sequence packing
- 6LoRA in depth: rank, alpha, target modules, and what the adapter actually learns
- 7QLoRA and quantized training: fitting larger models on smaller GPUs
- 8Hyperparameters that matter: learning rate, schedule, epochs, and early stopping
- 9Preference tuning with DPO and where RLHF is and is not worth the complexity
- 10Evaluation: task metrics, held-out sets, and measuring against the base model honestly
- 11Catastrophic forgetting: detecting capability regressions outside the target task
- 12Serving: adapter merging, multi-adapter inference, quantization, and throughput
- 13Lifecycle: versioning, reproducibility, and re-tuning when the base model changes
Learning Outcomes
- Make a defensible decision about whether fine-tuning is the right tool for a given problem
- Construct and validate a fine-tuning dataset, and recognize the flaws that ruin results
- Run LoRA and QLoRA fine-tunes on constrained GPU budgets
- Select hyperparameters deliberately instead of copying a blog post
- Evaluate a fine-tuned model against its base, including regressions outside the target task
- Serve fine-tuned models efficiently with adapters and quantization
- Plan the maintenance cost of a fine-tuned model over its life
What You Will Build
- A fine-tuned open-weight model adapter trained on a dataset you built
- An evaluation report comparing the tuned model against the base on task and general capability
- A written decision record justifying fine-tuning over prompting or RAG
- A serving configuration with measured throughput and memory footprint
Frequently Asked Questions
- Should we fine-tune or use RAG?
- They solve different problems. RAG supplies facts the model does not have; fine-tuning changes how the model behaves. If your issue is that the model does not know something, fine-tuning is the wrong tool and will teach it to sound confident rather than to be correct. If your issue is output format, tone, or domain vocabulary, fine-tuning helps. Plenty of systems need both, and the course covers combining them.
- How much data do we need?
- Far less than most teams expect, and quality dominates quantity. For supervised fine-tuning on a well-defined task, one to five thousand high-quality examples is often sufficient, and a curated thousand frequently outperforms a noisy fifty thousand. The course covers how to estimate the requirement for your task and how to tell when more data has stopped helping.
- What GPU do we need for the labs?
- Labs are built around QLoRA and run on a single 24 GB GPU, provisioned in the cloud lab environment so nothing is needed locally. The memory arithmetic is taught explicitly so you can size hardware for models larger than the lab uses.
- Do you cover RLHF?
- We cover DPO in depth because it is practical for most teams, and we cover RLHF conceptually along with an honest account of the infrastructure and annotation cost that makes full RLHF rarely worthwhile outside model labs.
- What happens when the base model gets updated?
- Your adapter is tied to specific base weights and does not carry forward. Planning for re-tuning, keeping the dataset and training config reproducible, and deciding whether an upgrade is worth the re-run is a maintenance cost most teams overlook. The course covers it explicitly.
Related Courses
NLP with Transformers
Understand transformer architecture properly, then apply it: tokenization, attention, embeddings, and fine-tuning encoder models for classification and extraction.
Deep Learning with PyTorch
Build, train, and debug neural networks in PyTorch: autograd, training loops, regularization, transfer learning, and the practices that make results reproducible.
Building Production-Ready RAG Systems
Take a RAG prototype to production: evaluation harnesses, incremental indexing, caching, latency budgets, and the failure modes that only appear at scale.
ML Model Deployment
Serve models reliably: inference APIs, batching, autoscaling, GPU utilization, model optimization, and hitting a latency target without overspending.
Ready to Get Started?
Contact us to schedule training for your team or inquire about upcoming sessions.