Deploying ML models with FastAPI : A Practical Guide

Introduction

If you are new to machine learning and have just trained a model with 95% accuracy, you may wonder how to make it available for others to test or integrate into their applications. In this tutorial, we’ll guide you step-by-step on how to deploy a machine learning model using FastAPI, Docker, and Heroku.

Dataset

First, let’s start with training a news classification model that can distinguish between fake vs real news. We’ll use the fake-and-real-news-dataset.

There are 23481 fake news and 21417 real news in the dataset. The data fields include title, text, subject, and date.

Sample Fake News

Sample Real News

We’ll need labels to distinguish between fake and real news before merging them.

# Add flag to dintinguish between fake and real
fake_news_df['label'] = '1'
real_news_df['label'] = '0'

We’ll merge fake and real news and drop columns that are unnecessary.

# Merge fake and real news
df_merged = pd.concat([fake_news_df,real_news_df])

# Drop unnecessary columns
df_merged = df_merged.drop(['subject','date'],axis=1)

88After Merging Two Dataframes

Create An SVC Model

Now that everything is ready, let’s train a model that can classify real vs fake news.

We’ll use a LinearSVC which stands for “Support Vector Classification”, a model based on SVMs.

Let’s check the confusion matrix of the model.

Confusion Matrix for the SVC model

This model provides us with 99.6% accuracy, which is quite good for our use case. However, the model may overfit this dataset because the news is from 2017 and may not accurately represent current events or other subjects not present in the dataset.

We could train a BERT model with more diverse data for better generalization, but the computation cost would be too high for a simple deployment. Therefore, we’ll use this relatively simple yet effective SVC model.

Now, let’s save the model for future use.

Deployment: FastAPI to create an API

To deploy this model, we need FastAPI, Uvicorn, Docker, and a Heroku account. We’ll create an API with FastAPI, and Uvicorn will serve as the ASGI web server. We will containerize the codebase with Docker and then deploy it to Heroku. So, make sure you have all the packages and software set up.

This will be the folder structure for the project.

Project Folder Structure

In app.py, let’s import the necessary packages and define a Pydantic model that defines the structure of input data. The “NewsInput” model will be used to parse and validate the request body data.

from fastapi import FastAPI
import pickle
from pydantic import BaseModel

# Define Pydantic model for request body data
class NewsInput(BaseModel):
    text: str

Then load the trained model and create a function to classify input texts.

# Load the classification model
model_path = "./models/trained_model_1.0.pkl"
news_classification_dict = {'0':'real','1':'fake'}

with open(model_path,'rb') as file:
    model = pickle.load(file)
    
# Function to classifiy the text
def classify_news(text: str) -> str:
    # Convert input text to a list
    sentences = [text]
    # Inference
    predictions = model.predict(sentences)
    result = news_classification_dict[str(predictions[0])]
    return result

Create FastAPI instance and define the endpoint as “/classify”.

@app.post(‘/classify’) decorator will create a POST endpoint at the ‘/classify’ URL path.

async def classify_text(news_input: NewsInput) : this function is an asynchronous function that handles POST requests to the endpoint and it will accept request body type of NewsInput.

Extract the text from request body : text = news_input.text

And call the classify_news function to classify the input and return the json result.

You can test the API with “uvicorn app:app — reload” cmd. If you are running the app on localhost the host address will be http://127.0.0.1 and it will run at port 8000. In the example below, I tested with a fake news paragraph generated from ChatGPT.

Testing the API from localhost

You can see that it can correctly classify the fake news paragraph.

You can also specify the host and port.

“uvicorn app:app — host <host> — port <port>”

Deployment : Docker and Heroku

Now that the app is running in localhost, we can move towards the next step. To create a docker image and deploy it with Heroku. To create a docker image, we need to write a Dockerfile which is a text document containing all the commands that we need to create an image.

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9 : This line specifies the base python image that we will use for the container. It’s a pre-configured base image that is optimized for FastAPI, Uvicorn and Gunicorn.

COPY ./requirements.txt /app/requirements.txt : This line copy the requirements.txt from local directory to the container.

RUN pip install — no-cache-dir — upgrade -r /app/requirements.txt : This line run pip install cmd to install the packages listed in requirements.txt in the container.

COPY ./app /app/ : This line copy the app directory from local directory to the Docker container.

CMD [“uvicorn”, “app:app”, “ — host”, “0.0.0.0”, “ — port”, “5000”] : This specify the cmd to run the app. It will launch our FastAPI using uvicorn at host : “0.0.0.0” and port “5000”.

You can build the image using “docker build -t <container-name> . ”.

You can list the image with “docker images” cmd.

After building the image, we can deploy it in Heroku.

First, login to Heroku with “heroku login” (you will need to install Heroku cli).

You will also need to add a “heroku.yml”, in the repository.

This basically specifies that the “web” process of the application to be built using the instructions provided int the Dockerfile.

And create a Heroku app “heroku create <app-name>”.

It will give you a URL for your app. After deploy the app, you can call to the API with that URL. Just a couple more cmds and the API will be up and running

“heroku git:remote <app-name>” cmd to set up git remote for the Heroku app.

“heroku stack:set container” to allow you to deploy the application using a Docker container.

And finally, “git push heroku main” to deploy your application.

You can then test the endpoint using the URL generated from the previous cmd.

After testing don’t forget to delete the app to reduce unnecessary costs.
“heroku apps:destroy — app <app-name>”.

So, that is pretty much it. I hope you find this tutorial useful and empowers you to take your models to production. Feel free to share your thoughts or ask any questions in the comments.
See you soon. (hopefully 😁)