Deploy yolov5 model in a few simple clicks

The second part of the two-part series tutorial shows how to deploy yolov5 which is already trained.

Last update: February 2022

Introduction

The following article presents how to deploy yolov5. It is the second part of the series in which we show you how to take your AI startup from zero to one. Previously, we’ve learned how to train yolov5 model designed for the Object Detection task. Therefore, in this part, we will learn how to deploy yolov5 model on production and by deployment, we mean serving a model in the form of scalable API.

If you want to skip the preparation part just go to Deploy yolov5 section. You can use the following repository with the prepared model.

Deployment Process

Before we learn how to deploy yolov5 model, let’s explore quickly the deployment process. In general, it consists of four stages. Specifically, creating a web service with Flask, recreating the environment in Docker, setting up an infrastructure, and deploying the model to a cloud provider like Google Cloud or AWS.

In fact, each of these steps requires highly technical knowledge. Even if you succeed to deploy yolov5 model, you will most likely find that it will work terribly in production. Thus your solution won’t be scalable to the increased traffic and the costs of the cloud will not be optimized. The reason is that production deployment requires explicitly programmed scalable API, as well as infrastructure and model monitoring.

Therefore in this tutorial, I will show you how to streamline this process and deploy a PyTorch model with one tool called Syndicai in a few simple clicks.

Prepare a model before you deploy yolov5

Before we deploy a yolov5 model we need to prepare it so that the platform will be able to serve in a scalable way. Therefore, in this section, we will focus only on two things - git repository, and configuration files. Of course, as was mentioned before, the model needs to be trained in order to deploy in production.

Create a GitHub repository for the model

To clarify, the Syndicai platform will allow us to deploy yolov5 by connecting a git repository in the first place. For that purpose, we need to create a git repository with our yolov5 model. Normally, we would start a new project, but since the model is already on GitHub, we can fork the repository. This will copy all the codes to our account.

As you probably noticed, there are no model weights in the repository. However, for purpose of the tutorial, we will use the weights posted by the authors of the yolov5.

In general, it is not a good practice to place model weights in the repository. The most common approach is to upload them to external storage and download them during the deployment process.

We already have the repository, but we have to spicy it up a little bit so that the Syndicai can recognize the model.

Add configuration files to deploy yolov5 via Syndcai

In order to deploy yolov5 via Syndicai, the repository needs to have syndicai.py and requirements.txt.

requirements.txt

The first file requirements.txt is already in the forked repository. It contains all the required packages needed to recreate the model’s environment.

Cython==0.29.17
matplotlib==3.2.2
numpy==1.18.5
opencv-python==4.2.0.34
Pillow==7.1.2
PyYAML==5.3
scipy==1.4.1
torchvision==0.8.1
tqdm==4.41.0
seaborn==0.11.1
pandas==1.0.3
torch==1.7.0
tensorboard==2.1

syndicai.py

Furthermore, we need to create syndicai.py. To clarify, the file consists of a PythonPredictor class with predict method and constructor. As the name suggests this will instruct the Syndicai on how to make a prediction.

The constructor __init__ takes one argument config. In general, the argument is not important for us right now but is necessary nevertheless. The constructor is also the best place to initialize your weights:

def __init__(self, config):
    self.model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

Next, let’s take a closer look at the predict function. It has one required argument payload which is a dictionary. The REST API takes a JSON file which is later transformed into a dictionary that goes into the predict function.

Since we are operating with images we need to deliver an image to a mode. We can encode them using the base64 format. However, we could also input a URL to an image.

When we deploy yolov5 we will test the model with the following sample input JSON.

{
    "url": "https://images.pexels.com/photos/2083866/pexels-photo-2083866.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260"
}

Below you will see the code responsible for unpacking input and packing the output. You can find the full code in the prepared yolov5 repository.

def predict(self, payload):
    """
    Called once per request. Preprocesses the request payload (if necessary), 
    runs inference, and postprocesses the inference output (if necessary).
    Args:
        payload: The request payload
    Returns:
        Prediction or a batch of predictions.
    """

    # Convert url image to PIL format
    img = url_to_img(payload["url"])

    # Run a model
    results = self.model(img)

    # Draw boxes
    boxes = results.xyxy[0].cpu().numpy()
    box_img = draw_box(img, boxes)

    # Return an image in the base64 format
    return img_to_bytes(box_img)

In addition to syndicai.py we can add helpers.py. It consists of a couple of helper functions that help to operate with images. See the repository.

Test the model locally

You can always test the model locally by adding a run.py to your repository. We will not explore this script in detail. However, you can have a look at the file in the repository.

Deploy yolov5

Finally, we will deploy yolov5 via Syndicai. We will achieve this by connecting a git repository to the platform, creating a deployment, and validating the model.

Connect a git repository to the platform

Firstly, let’s log in to the Syndicai platform. Logged in you will the dashboard, so go to Models and click Add model. You will see the simple form asking you for the repository URL and the path to the model. A git repository is “https://github.com/marcin-laskowski/yolov5”, while the path is blank since the model is placed in the root directory. Please have a look at the picture below.

As soon as you confirm the form by clicking Add, you will be redirected to the Model Profile.

Create a deployment

In the Model Profile click Deploy. Later fill the form with the name “Production”, and the branch “master”. Note, that if your branch is main, then place “main”.

Consequently, click Add to deploy yolov5.

You will be redirected to Deployment Profile. The platform will automatically create a new release of the deployment. You can see that by visiting the Releases tab. If you click on the recent release e.g. #1 you will see logs of that release.

Release logs cover building and starting phases. The building phase covers wrapping a model with the webservice and docker container. While a starting phase covers serving a docker container in the cloud.

Validate the deployment

To test out the deployment, go to the Deployment Profile. In the Overview tab scroll down to Validate & Integrate section.

/pic: validate & integrate

Edit a box with the sample model data. You can paste the following input code.

{
    "url": "https://images.pexels.com/photos/2083866/pexels-photo-2083866.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260"
}

The result is displayed in the base64 format. Therefore, after conversion, it should look as follows.

Conclusion on how to deploy yolov5

In this tutorial, we’ve learned how to deploy yolov5 model using the Syndicai platform. We were able to streamline the process that would normally require specialized DevOps. Moreover, the solution is scalable, secure, and cost-efficient. This two-part series of articles has allowed us to go from zero to one.

You can always come back and recap on how to train yolov5.