Sitemap
Open in app

Sign in

Medium Logo
Write

Sign in

Getting Started with CI/CD in Machine Learning

Vishnu Sivan
13 min readApr 11, 2025

Continuous Integration (CI) and Continuous Deployment (CD) have long been essential practices in modern software development, enabling teams to integrate changes frequently, run automated tests, and deploy applications efficiently. While these methodologies were originally designed for traditional software, their value is increasingly evident in the world of machine learning (ML), where reproducibility, automation, and rapid iteration are just as critical.

In this article, we’ll explore how CI/CD can streamline your machine learning workflows — from training and evaluating models to deploying them seamlessly. Instead of relying on complex tools or platforms, we’ll keep things simple and accessible by using GitHub Actions, Makefile, CML (Continuous Machine Learning), and the Hugging Face CLI to build a fully automated ML pipeline.

Getting Started

Table of contents

Setting up the project

In this section, we will guide you through setting up your environment, building a CI/CD pipeline, and optimizing the entire workflow. The drug classifier model is trained using a scikit-learn pipeline with a Random Forest model, automate evaluation with CML, and deploy everything to the Hugging Face Hub. Once everything is set up, every code push to GitHub will automatically retrain the model, evaluate it, and update the app, model, and results on Hugging Face.

Step 1: GitHub Repository

To begin, create a new GitHub repository for your machine learning project. This repo will host your code, datasets and configuration files for automation.

  • Go to GitHub, click the “+” icon in the top right, and select “New repository”.
  • Enter a repository name and optional description.
  • Check “Add a README file”.
  • Set .gitignore to Python.
  • Click “Create repository”.
  • Copy the repository URL and run the following commands in your terminal to clone it:
git clone your-github-repo-url
  • Example:
git clone https://github.com/codemaker2015/CICD-for-Machine-Learning.git 
cd CICD-for-Machine-Learning

Step 2: Hugging Face Spaces

To begin, create a new Hugging Face Space for your machine learning project. This Space will host your web application, model files, and serve as the deployment endpoint for your CI/CD pipeline.

  • Go to Hugging Face and click on your profile picture in the top right corner. Select “New Space” from the dropdown.
  • Fill in the required details:
  • Space name: Choose a unique name for your Space
  • License: Select an appropriate license
  • SDK type: Choose Gradio or Streamlit depending on your app
  • Click “Create Space” to finish setup.

Step 3: Project Structure

Let’s set up the required folders and files before experimenting and building the pipeline.

Create app, data, model and results folders in your GitHub cloned repository.

3.1 App folder

The App folder is used to store all files related to the Hugging Face Space. It contains the web application script (drug_app.py), a README.md file with metadata for the Space, and a requirements.txt file listing the necessary Python packages.

Create the following files inside the app folder:

  • app.py: The main script for your classifier web app.
  • README.md: Contains metadata and a description for your Hugging Face Space. You can either download the README.md file directly from the Hugging Face Space you created earlier or use the sample content provided below.
---
title: Drug Classification
emoji: 💻
colorFrom: pink
colorTo: red
sdk: gradio
sdk_version: 5.23.3
app_file: app.py
pinned: false
license: apache-2.0
---
  • requirements.txt: Specifies the dependencies needed to run your app. Add the following packages to the requirements.txt file.
scikit-learn
skops

3.2 Data folder

Download the Drug Classification dataset from Kaggle, extract the contents, and move the CSV file into the data folder.

Drug Classification

This database contains information about certain drug types.

www.kaggle.com

3.3 Model and Results folder

Both the models and results folders will initially be empty. They will be automatically populated by the Python scripts during training and evaluation.

3.4 Repository files

  • Makefile: Defines command shortcuts for running scripts, making it easier to trigger processes in the GitHub Actions workflow.
  • requirements.txt: Lists all the dependencies required to set up the environment for CI workflow jobs. Add the following dependencies to the requirements.txt.
pandas
scikit-learn
numpy
matplotlib
skops
  • train.py: Contains the core Python logic to load and preprocess data, train and evaluate the model, and save both the trained model and performance metrics.

Your project folder should now look like this:

Training and Evaluating Drug Classification Model

In this section, we will experiment with Python code to process the data and train a model using a scikit-learn pipeline. After training, evaluate the model performance and save both the results and the trained model for later use.

Installing the dependencies

  • Create and activate a virtual environment by executing the following command.
python -m venv venv
source venv/bin/activate #for ubuntu
venv/Scripts/activate #for windows
  • Install pandas, scikit-learn, numpy, matplotlib, skops and black libraries using pip.
pip install pandas scikit-learn numpy matplotlib skops black

Loading the Dataset

Load the CSV file using Pandas, shuffle the rows using the sample() function to randomize the data, and then display the first three rows.

import pandas as pd

drug_df = pd.read_csv("data/drug.csv")
drug_df = drug_df.sample(frac=1)
print(drug_df.head(3))

Splitting train and test data

Define the independent variables (X) and dependent variable (y) then split the dataset into training and testing sets. This is essential for evaluating the model performance on unseen data.

from sklearn.model_selection import train_test_split

X = drug_df.drop("Drug", axis=1).values
y = drug_df.Drug.values

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=125
)

Building the Training Pipeline

Construct a data processing pipeline using ColumnTransformer, which performs the following operations:

  • Encodes categorical columns using OrdinalEncoder
  • Fills missing values in numerical columns using SimpleImputer
  • Scales the numerical columns using StandardScaler

After preprocessing, build a training pipeline that feeds the transformed data into a RandomForestClassifier.

from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OrdinalEncoder, StandardScaler

# Define categorical and numerical column indices
cat_col = [1, 2, 3]
num_col = [0, 4]

# Create a column transformer for preprocessing
transform = ColumnTransformer(
    transformers=[
        ("encoder", OrdinalEncoder(), cat_col),
        ("num_imputer", SimpleImputer(strategy="median"), num_col),
        ("num_scaler", StandardScaler(), num_col),
    ]
)

# Build the complete pipeline
pipe = Pipeline(
    steps=[
        ("preprocessing", transform),
        ("model", RandomForestClassifier(n_estimators=100, random_state=125)),
    ]
)

# Train the model
pipe.fit(X_train, y_train)

Evaluating the model

After training the model, evaluate its performance using two common metrics: accuracy and F1 score.

from sklearn.metrics import accuracy_score, f1_score

predictions = pipe.predict(X_test)
accuracy = accuracy_score(y_test, predictions)
f1 = f1_score(y_test, predictions, average="macro")

print("Accuracy:", str(round(accuracy * 100, 2)) + "%", "F1 Score:", round(f1, 2))

Saving the model and results

We will store the evaluation metrics and confusion matrix in the results/ folder. This helps in tracking performance over time, especially in CI/CD pipelines.

1. Save Accuracy and F1 Score to a Text File

with open("results/metrics.txt", "w") as outfile:
    outfile.write(f"Accuracy = {round(accuracy, 2)}, F1 Score = {round(f1, 2)}")

2. Save Confusion Matrix as an Image

import matplotlib.pyplot as plt
from sklearn.metrics import ConfusionMatrixDisplay, confusion_matrix
# Generate confusion matrix
cm = confusion_matrix(y_test, predictions, labels=pipe.classes_)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=pipe.classes_)
# Plot and save the confusion matrix
disp.plot()
plt.savefig("results/model_results.png", dpi=120)

3. Save and Load model using skops

We will use the skops Python package to save our entire pipeline including both the preprocessing steps and the trained model. With skops, model versioning and reproducibility become much easier in a CI/CD workflow.


import skops.io as sio

# Save the trained pipeline to a file
sio.dump(pipe, "model/drug_pipeline.skops")
Load the Model Pipeline

# Load the saved pipeline
loaded_pipe = sio.load("model/drug_pipeline.skops", trusted=True)

Creating train.py file

Here’s how you can structure your train.py file using the code snippets you've worked on. This script will handle the loading, training, evaluation, saving of the model and results in a modular way.

import pandas as pd
import matplotlib.pyplot as plt
import skops.io as sio

from sklearn.model_selection import train_test_split
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OrdinalEncoder, StandardScaler
from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, ConfusionMatrixDisplay

# Load and shuffle dataset
drug_df = pd.read_csv("data/drug.csv")
drug_df = drug_df.sample(frac=1)

# Train-test split
X = drug_df.drop("Drug", axis=1).values
y = drug_df["Drug"].values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=125)

# Define preprocessing and model pipeline
cat_col = [1, 2, 3]
num_col = [0, 4]

transform = ColumnTransformer([
    ("encoder", OrdinalEncoder(), cat_col),
    ("num_imputer", SimpleImputer(strategy="median"), num_col),
    ("num_scaler", StandardScaler(), num_col),
])

pipe = Pipeline(steps=[
    ("preprocessing", transform),
    ("model", RandomForestClassifier(n_estimators=100, random_state=125)),
])

# Train the model
pipe.fit(X_train, y_train)

# Make predictions and evaluate
predictions = pipe.predict(X_test)
accuracy = accuracy_score(y_test, predictions)
f1 = f1_score(y_test, predictions, average="macro")

# Save metrics
with open("results/metrics.txt", "w") as outfile:
    outfile.write(f"Accuracy = {round(accuracy, 2)}, F1 Score = {round(f1, 2)}")

# Save confusion matrix
cm = confusion_matrix(y_test, predictions, labels=pipe.classes_)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=pipe.classes_)
disp.plot()
plt.savefig("results/model_results.png", dpi=120)

# Save model
sio.dump(pipe, "model/drug_pipeline.skops")

Building Your Machine Learning CI Pipeline

In this section, we will explore how to use CML, Makefile, and GitHub Actions to automate model training, evaluation, and version control for our machine learning project.

CML

Continuous Machine Learning (CML) is an open-source tool for integrating CI into ML projects. We will use the iterative/setup-cml GitHub Action to automate model evaluation reporting. On every push, it generates a report with performance metrics and a confusion matrix under the commit and sends an email notification.

Creating update branch

We’re generating the evaluation report, but currently, the model and results aren’t being versioned. To track these changes properly, we’ll create a new branch called “update” and push the updated model and results to it.

To create the “update” branch:

  • Click on the branch dropdown (where it says main)
  • Type “update” in the search box
  • Select “Create branch: update from main” to finalize the creation.

Makefile

A Makefile contains command sets that can automate tasks like preprocessing, training, testing, and deploying. It helps simplify the CI workflow by bundling related commands, keeping the GitHub Actions file clean and modular.

  • Add the following content to the Makefile.
install:
 pip install --upgrade pip &&\
  pip install -r requirements.txt

format: 
 black *.py 

train:
 python train.py

eval:
 echo "## Model Metrics" > report.md
 cat ./results/metrics.txt >> report.md
 
 echo '\n## Confusion Matrix Plot' >> report.md
 echo '![Confusion Matrix](./results/model_results.png)' >> report.md
 
 cml comment create report.md
  
update-branch:
 git config --global user.name $(USER_NAME)
 git config --global user.email $(USER_EMAIL)
 git commit -am "Update with new results"
 git push --force origin HEAD:update

hf-login: 
 pip install -U "huggingface_hub[cli]"
 git pull origin update
 git switch update
 huggingface-cli login --token $(HF) --add-to-git-credential

push-hub: 
 huggingface-cli upload codemaker2015/Drug-Classification ./app --repo-type=space --commit-message="Sync App files"
 huggingface-cli upload codemaker2015/Drug-Classification ./model model --repo-type=space --commit-message="Sync Model"
 huggingface-cli upload codemaker2015/Drug-Classification ./results metrics --repo-type=space --commit-message="Sync Model"

deploy: hf-login push-hub

all: install format train eval update-branch deploy
  • After we make the necessary changes, commit them, and push the updates to the remote GitHub repository.
git add .
git commit -m "code integration"
git push origin main

GitHub Actions

To automate training and evaluation, we will create a GitHub Actions workflow.

  • Go to the “Actions” tab in your GitHub repository.
  • Click on “set up a workflow yourself.”
  • Rename the default main.yml file to ci.yml.
  • Start by defining the name of the workflow.
  • Set the trigger so that it runs on every push or pull request to the main branch or through manual dispatch.
  • Define the environment by using the latest Ubuntu runner.
  • Set up and activate the GitHub Actions we need for this CI workflow.
  • Use make commands to add different steps like installing dependencies, training, formatting, and evaluating.
  • Commit your changes to trigger the workflow — GitHub Actions will execute each step sequentially.
  • Provide a GitHub Token to the CML job via repository secrets (e.g., secrets.GITHUB_TOKEN).
  • Add the following code into your ci.yml file:
name: Continuous Integration
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
  workflow_dispatch:
  
permissions: write-all
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: iterative/setup-cml@v2
      - name: Install Packages
        run: make install
      - name: Format
        run: make format
      - name: Train
        run: make train
      - name: Evaluation
        env:
          REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: make eval
      - name: Update Branch
        env:
          NAME: ${{ secrets.USER_NAME }}
          EMAIL: ${{ secrets.USER_EMAIL }}
        run: make update-branch USER_NAME=$NAME USER_EMAIL=$EMAIL

Building Your Machine Learning CD Pipeline

In this section, we will explore how to automate the deployment of both the model and the application. This includes pulling the updated model and app files from the update branch, logging into the Hugging Face CLI using a token, pushing the necessary files, and ultimately deploying the application.

Building the Gradio App

To deploy our model and make it accessible, we’ll create a Gradio app with the following components:

  • Load the scikit-learn pipeline and trained model.
  • Define a Python function to predict drug labels based on user input.
  • Design the input interface using sliders for numerical values and radio buttons for categorical inputs.
  • Add sample inputs to quickly test the model’s functionality.
  • Provide metadata such as title for the application and brief description highlighting its features and purpose.

Add the following code to app.py file inside the app folder.

import gradio as gr
import skops.io as sio
import warnings
from sklearn.exceptions import InconsistentVersionWarning

# Suppress the version warnings
warnings.filterwarnings("ignore", category=InconsistentVersionWarning)

# Explicitly specify trusted types
trusted_types = [
    "sklearn.pipeline.Pipeline",
    "sklearn.preprocessing.OneHotEncoder",
    "sklearn.preprocessing.StandardScaler",
    "sklearn.compose.ColumnTransformer",
    "sklearn.preprocessing.OrdinalEncoder",
    "sklearn.impute.SimpleImputer",
    "sklearn.tree.DecisionTreeClassifier",
    "sklearn.ensemble.RandomForestClassifier",
    "numpy.dtype",
]
pipe = sio.load("./model/drug_pipeline.skops", trusted=trusted_types)


def predict_drug(age, sex, blood_pressure, cholesterol, na_to_k_ratio):
    """Predict drugs based on patient features.

    Args:
        age (int): Age of patient
        sex (str): Sex of patient
        blood_pressure (str): Blood pressure level
        cholesterol (str): Cholesterol level
        na_to_k_ratio (float): Ratio of sodium to potassium in blood

    Returns:
        str: Predicted drug label
    """
    features = [age, sex, blood_pressure, cholesterol, na_to_k_ratio]
    predicted_drug = pipe.predict([features])[0]

    label = f"Predicted Drug: {predicted_drug}"
    return label


inputs = [
    gr.Slider(15, 74, step=1, label="Age"),
    gr.Radio(["M", "F"], label="Sex"),
    gr.Radio(["HIGH", "LOW", "NORMAL"], label="Blood Pressure"),
    gr.Radio(["HIGH", "NORMAL"], label="Cholesterol"),
    gr.Slider(6.2, 38.2, step=0.1, label="Na_to_K"),
]
outputs = [gr.Label(num_top_classes=5)]

examples = [
    [30, "M", "HIGH", "NORMAL", 15.4],
    [35, "F", "LOW", "NORMAL", 8],
    [50, "M", "HIGH", "HIGH", 34],
]


title = "Drug Classification"
description = "Enter the details to correctly identify Drug type?"
article = "A Beginners Guide to CI/CD for Machine Learning. It teaches how to automate training, evaluation, and deployment of models to Hugging Face using GitHub Actions."


gr.Interface(
    fn=predict_drug,
    inputs=inputs,
    outputs=outputs,
    examples=examples,
    title=title,
    description=description,
    article=article,
    theme=gr.themes.Soft(),
).launch()

Add the following dependencies to the requirements.txt inside the app folder.

scikit-learn
skops
gradio

CD workflow

To make our workflow fully CI/CD compliant, we need to create another file named cd.yml, similar to the existing ci.yml file. Once the CI pipeline completes successfully, it will trigger the cd.yml workflow using the on.workflow_run parameter. This deployment workflow will set up the environment and execute the make deploy command, using the Hugging Face token to push the latest model and application updates to the Hugging Face Hub.

  • Go to GitHub actions and create a workflow named as cd.yml.
  • Add the following code to it.
name: Continuous Deployment
on:
  workflow_run:
    workflows: ["Continuous Integration"]
    types:
      - completed

  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Deployment To Hugging Face
        env:
          HF: ${{ secrets.HF }}
        run: make deploy HF=$HF

Setting up repository secrets

To commit and push changes using Git, you need to configure a username and email. While you can set these directly, it’s recommended to use GitHub Secrets for better security. Additionally, the CD pipeline requires a Hugging Face token to deploy the application to the Hugging Face Hub.

Follow these steps to securely add the necessary credentials to your GitHub repository using Secrets.

  • Go to your repository Settings and click on “Secrets and variables” under the Security section.
  • Select “Actions”, then click the green “New repository secret” button.
    Add a name and value — this works like setting an environment variable on your local machine.
  • To generate a Hugging Face token, click on your profile picture and select “Settings”.
  • Navigate to “Access Tokens”, then click “New Token” and ensure it has write permissions.
  • Copy the token and add it as a repository secret in the same way as you did for the Git username and email.
  • After we make the necessary changes, commit them, and push the updates to the remote GitHub repository. Note that we have added a few GitHub Actions, so make sure to pull the latest changes from the remote repository before pushing your local updates.
git add .
git commit -m "gradio code added"
git pull origin main
git push origin main
  • Once you push the changes, GitHub Actions will be triggered automatically to run the CI/CD pipelines. Alternatively, you can run them manually by clicking the Re-run all jobs button.
  • You can monitor live logs for each step by selecting the run option in the workflow build. Once the files are successfully uploaded to the Hugging Face server, the corresponding Space will begin setting up the environment. Shortly after, the application will launch and start running.

Project Resources

Continuous Integration
Continuous Deployment
Machine Learning
Python
Beginners Guide

--

--

Vishnu Sivan
Vishnu Sivan

Written by Vishnu Sivan

1K followers
181 following

Try not to become a man of SUCCESS but rather try to become a man of VALUE

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Rules

Terms

Text to speech