Large Language Models

Fine Tuning

Llama 3

Google Colab

Unsloth

From Pretrained to Purposeful: Fine-Tuning LLaMA 3.2 Made Easy with Unsloth

Large language models like LLaMA and Mistral have accelerated open-source innovation, but their size often makes them difficult to fine-tune or deploy on everyday hardware. To address this, smaller models such as TinyLlama-1B, Microsoft Phi-2, and Alibaba Qwen-3B have emerged — offering strong performance in a much smaller footprint.

Still, fine-tuning is essential to adapt any base model to specific tasks like support chat, summarization, or domain-specific Q&A. Traditionally, this requires high memory and compute, putting it out of reach for many developers.

Unsloth solves this challenge by enabling efficient fine-tuning on modest hardware. It uses LoRA (Low-Rank Adaptation) to reduce the number of trainable parameters, lowering memory consumption significantly. Key to Unsloth’s performance is the integration of BitsAndBytes, a library that enables 4-bit and 8-bit quantization. Combined with LoRA, it drastically reduces memory usage and accelerates training — making it possible to fine-tune models on GPUs with limited VRAM.

In this guide, we’ll fine-tune LLaMA 3.2 (3B) using Maxime Labonne’s FineTome-100k dataset in ShareGPT format, demonstrating a practical and efficient setup for real-world fine-tuning without the need for expensive infrastructure.

Getting Started

Table of contents

• What is LLM fine-tuning
• What is Unsloth
• Getting Started with Fine-Tuning LlaMa 3.2
• Setting up the environment
• Create and Configure Your Colab Notebook
• Add Hugging Face Access Token (Optional but Recommended)
• Install Unsloth and Required Dependencies
• Loading the Model and Tokenizer
• Applying LoRA Adapters for Efficient Fine-Tuning
• Preparing the Training Dataset
• Standardizing the Dataset Format
• Loading the dataset
• Formatting Prompts
• Setting Up and Configuring the Trainer
• Training Only on Assistant Responses
• Inference — Generating responses
• Saving and Loading the Fine-Tuned Model
• Load the LoRA Adapters for Inference

What is LLM Fine-Tuning

Fine-tuning is the process of adapting a pre-trained large language model (LLM) to perform better on a specific task or domain. While pre-trained models are trained on massive amounts of general-purpose data, they often fall short in specialized use cases. Fine-tuning bridges this gap by training the model further on curated, domain-specific datasets.

For instance, while a base LLM might perform well on single-turn question-answering tasks, it may struggle with multi-turn conversations typically expected from chatbot systems. To handle such scenarios, the model needs exposure to dialogue-format datasets — something achieved through fine-tuning.

Fine-tuning allows developers to mold general LLMs into custom “avatars” suited for various tasks such as legal document summarization, healthcare Q&A, or multilingual support. The effectiveness of fine-tuning largely depends on the quality of the dataset, the capabilities of the base model, and the method of fine-tuning used.

Common Fine-Tuning Techniques

• Full Fine-Tuning: This traditional method updates all the parameters of the model. While effective, it requires significant computational resources and memory, making it less feasible for large models or limited hardware setups.
• LoRA (Low-Rank Adaptation): LoRA introduces small trainable matrices (adapters) into the model and only updates them, freezing the rest of the model weights. This reduces compute requirements and speeds up training — ideal for fine-tuning large models on consumer GPUs.
• QLoRA (Quantized LoRA): QLoRA goes a step further by applying LoRA to a quantized version of the model. The model weights are first reduced to 4-bit or 8-bit precision using libraries like BitsAndBytes, drastically lowering memory consumption while retaining near-original performance.
• Adapter Tuning: Adapter tuning inserts additional layers (adapters) into the network without modifying the original model weights. Like LoRA, it allows task-specific tuning with low resource usage and easy parameter sharing.
• Prompt-Tuning / Prefix-Tuning: Instead of changing model parameters, this method learns a small prompt or prefix that conditions the model to perform a specific task. It’s lightweight and especially useful when storage or compute resources are constrained.

What is Unsloth

Unsloth is an open-source framework purpose-built for fast and efficient fine-tuning of large language models (LLMs). It provides an optimized training backend, making fine-tuning possible even on limited hardware setups by drastically improving training speed and memory efficiency.

At its core, Unsloth integrates custom Triton kernels and a manual backpropagation engine to accelerate training. This results in significant speedups — up to 2x faster than traditional fine-tuning pipelines — without compromising on performance.

Unsloth’s compatibility with QLoRA and BitsAndBytes further enhances its resource efficiency, making it one of the best frameworks for developers looking to fine-tune LLMs quickly and affordably.

Unsloth supports a wide range of popular models, including the latest LLaMA 3.2, Mistral, Phi, and Gemma variants. Most of these models are available in 4-bit quantized format (bnb-4bit), making them ideal for fine-tuning on consumer GPUs with limited VRAM.

Currently Supported Models (4-bit):

• LLaMA 3.1 & 3.2: Meta-Llama-3.1-8B-bnb-4bit, Meta-Llama-3.1-8B-Instruct-bnb-4bit, Meta-Llama-3.1-70B-bnb-4bit, Meta-Llama-3.1-405B-bnb-4bit, Llama-3.2-1B-bnb-4bit, Llama-3.2-1B-Instruct-bnb-4bit, Llama-3.2-3B-bnb-4bit, Llama-3.2-3B-Instruct-bnb-4bit, Llama-3.3-70B-Instruct-bnb-4bit
• Mistral: Mistral-Small-Instruct-2409, mistral-7b-instruct-v0.3-bnb-4bit
• Phi: Phi-3.5-mini-instruct, Phi-3-medium-4k-instruct
• Gemma: gemma-2-9b-bnb-4bit, gemma-2-27b-bnb-4bit

Getting Started with Fine-Tuning LlaMa 3.2

Fine-tuning large language models, even smaller variants, is a compute-intensive task. It typically requires a machine with at least 10–15 GB of VRAM. Fortunately, free cloud platforms like Google Colab and Kaggle Notebooks offer accessible environments equipped with GPUs — ideal for getting started without local setup.

For this hands-on guide, we will be using Google Colab with a T4 GPU.

Setting up the environment

Step 1: Create and Configure Your Colab Notebook

1. Open Google Colaboratory and sign in with your Google account.
2. Create a new notebook by clicking on + New Notebook.
3. Navigate to Runtime → Change runtime type.

• Set Hardware Accelerator to GPU.
• Choose T4 GPU (recommended for this tutorial).
• Click Save.

💡 Tip: You can also run this setup on Kaggle by enabling GPU under “Accelerator” in notebook settings.

Step 2: Add Hugging Face Access Token (Optional but Recommended)

If you’re pulling models from Hugging Face, you’ll need an access token:

1. In the left sidebar, select the 🔑 Secrets tab.
2. Add a new secret:

• Key: HF_TOKEN
• Value: Your Hugging Face access token

3. Generate a write token from your Hugging Face profile settings → Create new token → Select write → Provide token name → Click on Create token.

Step 3: Install Unsloth and Required Dependencies

Use the following script to install all necessary packages:

!pip install --no-deps bitsandbytes accelerate xformers==0.0.29.post3 peft trl triton cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf "datasets>=3.4.1" huggingface_hub hf_transfer
!pip install --no-deps unsloth

Loading the Model and Tokenizer

Load the LLaMA 3.2 model using Unsloth’s optimized loading utilities. For this tutorial, we will be working with the Llama-3.2-3B-Instruct-bnb-4bit variant, which is quantized for efficient fine-tuning on limited hardware.

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Llama-3.2-3B-Instruct-bnb-4bit",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

💡 Tip: If you’d like to fine-tune a different model, simply update the model_name variable with the desired model's name from Unsloth’s supported list.

Applying LoRA Adapters for Efficient Fine-Tuning

Low-Rank Adaptation (LoRA) enables efficient fine-tuning by updating only a small subset of the model’s parameters. This significantly reduces memory usage and accelerates training, making it ideal for resource-constrained environments.

model = FastLanguageModel.get_peft_model(
    model,
    r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 16,
    lora_dropout = 0, # Supports any, but = 0 is optimized
    bias = "none",    # Supports any, but = "none" is optimized
    # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
    random_state = 3407,
    use_rslora = False,  # Unsloth support rank stabilized LoRA
    loftq_config = None, # And LoftQ
)

Preparing the Training Dataset

Before starting the training process, we need to load and preprocess the dataset. In this guide, we will use Maxime Labonne’s FineTome-100k, a high-quality dataset formatted in ShareGPT-style multi-turn conversations.

You are free to use any dataset, but it must be structured correctly for the model to interpret the inputs properly. If your dataset isn’t already in the expected format, you will need to preprocess it accordingly. The Hugging Face Datasets documentation is a helpful resource for transforming and preparing datasets for fine-tuning.

Standardizing the Dataset Format

For LLaMA 3.x models, Unsloth expects conversations to follow a specific format — similar to:

<|begin_of_text|><|start_header_id|>user<|end_header_id|>
Hello!<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Hey there! How are you?<|eot_id|>

To ensure compatibility, we convert the ShareGPT format to the standard Hugging Face multi-turn format using fields like "role" and "content" (instead of "from" and "value").

To ensure compatibility with the LLaMA 3.x training pipeline, we use the standardize_sharegpt utility to convert datasets from ShareGPT-style to Hugging Face’s generic multi-turn conversation format.

For example, the original format:

{"from": "system", "value": "You are an assistant"}
{"from": "human", "value": "What is 2+2?"}
{"from": "gpt", "value": "It's 4."}

Is transformed into the standardized format:

{"role": "system", "content": "You are an assistant"}
{"role": "user", "content": "What is 2+2?"}
{"role": "assistant", "content": "It's 4."}

This standardized structure ensures compatibility with Unsloth’s get_chat_template() function and avoids tokenization or formatting issues during training. It’s a crucial preprocessing step for models that expect role-based dialogue formatting.

Loading the dataset

To begin fine-tuning, we first need to load the dataset into our environment. In this tutorial, we use the FineTome-100k dataset curated by Maxime Labonne, which contains high-quality multi-turn conversations in ShareGPT format.

from datasets import load_dataset
dataset = load_dataset("mlabonne/FineTome-100k", split = "train")

Formatting Prompts

After preparing the dataset, the next step is to structure the data using the appropriate chat format expected by the model. In this case, we apply the LLaMA 3.1 chat template using Unsloth’s get_chat_template() function. This function configures the tokenizer to format prompts in the LLaMA-style conversational structure, ensuring the model can effectively process and learn from multi-turn dialogues during fine-tuning.

from unsloth.chat_templates import get_chat_template
from unsloth.chat_templates import standardize_sharegpt

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "llama-3.1",
)

def formatting_prompts_func(examples):
    convos = examples["conversations"]
    texts = [tokenizer.apply_chat_template(convo, tokenize = False, add_generation_prompt = False) for convo in convos]
    return { "text" : texts, }

dataset = standardize_sharegpt(dataset)
dataset = dataset.map(formatting_prompts_func, batched = True,)

To verify that the dataset is correctly structured for fine-tuning with the LLaMA 3.1 format, it’s useful to inspect both the original conversation format and the formatted text version.

# View the original conversation format
print(dataset[5]["conversations"])
# View the same item in the formatted text format
print(dataset[5]["text"])

Setting Up and Configuring the Trainer

With the dataset and model prepared, the next step is to configure the fine-tuning process using Hugging Face’s SFTTrainer. This trainer simplifies fine-tuning by handling essential tasks such as tokenization, batching, gradient accumulation, and optimization. It is fully compatible with Unsloth, enabling efficient training with reduced VRAM consumption and improved speed.

For this tutorial, the training is limited to 60 steps for demonstration purposes. However, for a complete fine-tuning run, you can set num_train_epochs=1 and max_steps=None to train over the entire dataset.

from trl import SFTTrainer
from transformers import TrainingArguments, DataCollatorForSeq2Seq
from unsloth import is_bfloat16_supported

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),
    dataset_num_proc = 2,
    packing = False, # Can make training 5x faster for short sequences.
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        # num_train_epochs = 1, # Set this for 1 full training run.
        max_steps = 60, # Limit training steps to 60 (for quick testing)
        learning_rate = 2e-4,
        fp16 = not is_bfloat16_supported(),
        bf16 = is_bfloat16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs", # Directory to save model checkpoints
        report_to = "none", # Use this for WandB etc
    ),
)

Training only on Assistant Responses

To make the training process more efficient and focused, we configure the model to learn only from the assistant’s responses, while ignoring the user’s inputs during loss computation. This approach helps the model better understand how to generate high-quality replies without being penalized for user input patterns.

Unsloth provides a convenient utility for this purpose: train_on_responses_only from unsloth.chat_templates.

from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
    trainer,
    instruction_part="<|start_header_id|>user<|end_header_id|>\n\n",     # Marks user input
    response_part="<|start_header_id|>assistant<|end_header_id|>\n\n",   # Marks assistant response
)
# Begin training
trainer_stats = trainer.train()

This setup ensures that the model is optimized solely on the assistant’s outputs. While training loss may decrease gradually, that’s expected — especially when using a small number of training steps. In this example, we limited training to 60 steps for quick experimentation.

💡 Tip: For better performance, it’s recommended to fine-tune the model for 2–3 epochs on large datasets or 3–5 epochs on smaller ones. Aim for 500+ training steps at minimum, and ideally 1000+ steps if hardware resources permit.

Inference — Generating responses

Once fine-tuning is complete, the trained model is ready for inference — generating responses based on new inputs. To run inference, simply provide an instruction and input, leaving the output field blank. The model will generate a response accordingly.

For this example, we use the following decoding parameters:

• min_p = 0.1 – to ensure a level of sampling diversity
• temperature = 1.5 – to introduce controlled randomness in the output

Feel free to adjust these values to fine-tune response creativity and coherence based on your use case.

from unsloth.chat_templates import get_chat_template

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "llama-3.1",
)
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

messages = [
    {"role": "user", "content": "Continue the fibonnaci sequence: 1, 1, 2, 3, 5, 8,"},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True, # Must add for generation
    return_tensors = "pt",
).to("cuda")

outputs = model.generate(input_ids = inputs, max_new_tokens = 64, use_cache = True,
                         temperature = 1.5, min_p = 0.1)

# Decode the generated tokens into human-readable text
text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(text) 

Saving and Loading the Fine-Tuned Model

Once training is complete, you can save your fine-tuned model and tokenizer either locally or push them to the Hugging Face Hub.

Save Locally

model_name = "Llama32_fine_tuned"
model.save_pretrained(model_name)
tokenizer.save_pretrained(model_name)

This will store only the LoRA adapter weights, not the full base model.

Push to Hugging Face Hub

To make your fine-tuned adapters publicly or privately accessible:

model.push_to_hub("your_name/your_model_name")
tokenizer.push_to_hub("your_name/your_model_name")

Save the Full Model in GGUF Format (Optional)

To export the complete model (base + LoRA adapters) in the efficient GGUF format which is suitable for CPU inference, use the following method:

model.push_to_hub_gguf(model_name, tokenizer, quantization_method="q4_k_m")

This compresses the model using the q4_k_m quantization method, which helps reduce model size and boosts inference performance.

Load the LoRA Adapters for Inference

To use your saved LoRA adapters for inference:

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "Llama32_fine_tuned",  # Name of your fine-tuned model
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)
FastLanguageModel.for_inference(model)  # Enable optimized inference

Now generate responses using the tokenizer and the trained model:

messages = [
    {"role": "user", "content": "Describe a tall tower in the capital of France."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")
from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer, skip_prompt=True)
_ = model.generate(
    input_ids=inputs,
    streamer=text_streamer,
    max_new_tokens=128,
    use_cache=True,
    temperature=1.5,
    min_p=0.1
)

This will generate a response in real-time using your fine-tuned model.

Thanks for reading this article !!

Thanks Gowri M Bhatt for reviewing the content.

If you enjoyed this article, please click on the clap button 👏 and share to help others find it!

The full source code for this tutorial can be found here,

GitHub - codemaker2015/llama3.2-fine-tuning: This repository contains a step-by-step Colab notebook…

Google Colab — Finetuning LLMs

codemaker2015/Llama32_fine_tuned · Hugging Face

References

GitHub - unslothai/unsloth: Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train Qwen3, Llama 4…

Unsloth Docs | Unsloth Documentation