This tutorial will cover a fast and straightforward way to deploy a PyTorch model at scale without a docker and Kubernetes setup.

I've trained my model. What's next?

Have you ever wanted to deploy a Pytorch model to run it on a website or some mobile app? It is not an easy task as you probably noticed.

In the past, I was exactly in such a situation, trying different tools and resources to serve a sample Machine Learning model in the form of API. Meanwhile, I've experienced that most Machine Learning Deployment tutorials' primary focus is building a simple flask app, creating a docker, and setting up the cloud infrastructure. However, it is still time-consuming and challenging. Nevertheless, hundreds of hours spent building a simple one-page demo app that takes an image as an input and returns one word: Cat.

Since that time, I've managed to find a way to solve that problem in just three easy steps.

Let's look and deploy a PyTorch model (Check also How to deploy Keras model).

Step 1: Develop a model

In the first step, we need to have a trained model. For this purpose, we will use a pre-trained PyTorch YoloV5.

For those who don't know what, Yolo is a real-time object detection framework and stands for You Only Look Once, which means that the image passes only once through the Fully Convolutional Neural Network. Compared to previous versions, the V5 is much faster and smaller (only 27Mb).

deploy pytorch model
The input of the model (on the left) and output (on the right)

In order to run the algorithm locally, you can copy and paste a script from PyTorch Hub. The Torch library will automatically download a pre-trained model and sample images to perform the inference of a model.

import cv2
import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).fuse().autoshape() # for PIL/cv2/np inputs and NMS

# Images
for f in ['zidane.jpg', 'bus.jpg']: # download 2 images
print(f'Downloading {f}...')
torch.hub.download_url_to_file('' + f, f)
img1 ='zidane.jpg') # PIL image
img2 = cv2.imread('bus.jpg')[:, :, ::-1] # OpenCV image (BGR to RGB)
imgs = [img1, img2] # batched list of images

# Inference
results = model(imgs, size=640) # includes NMS

# Results # .show() results, .save() jpgs, or .print() to scree

Step 2: Deploy a Pytorch model

We have our basic script with the model, so now we can deploy it in the cloud.

For this purpose, we will use the Syndicai Platform, which helps developers deploy the AI model to production effortlessly without setting up an infrastructure. In addition, it takes care of security, scalability, monitoring. You don’t need to set up or configure anything. You need to prepare a model and connect your git repository and later the model will be deployed automatically. In the first place, we will prepare a model, then we will connect a git repository to the platform in order to finally deploy a pytorch model by creating the API.

deploy a pytorch model
AI Model Deployment: Traditional Approach vs. Syndicai

Prepare a model

During the model preparation process, you need to ensure the following files are in the git repository:

1st file: requirements.txt - file with all libraries and frameworks required to recreate model's environment.


2nd file: - main file with the PythonPredictor python class responsible for model prediction.

import torch
from PIL import Image
from helpers import draw_box, url_to_img, img_to_bytes

class PythonPredictor:

def __init__(self, config):
""" Download pretrained model. """
self.model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()

def predict(self, payload):
""" Run a model based on url input. """

# Inference
img = url_to_img(payload["url"])
results = self.model(img)

# Draw boxes
boxes = results.xyxy[0].numpy()
box_img = draw_box(img, boxes)

# Save image"sample_data/output.png", "PNG")

return img_to_bytes(box_img)

Those files are necessary to correctly build recreate the environment, build the infrastructure, and finally run the model in the form of a webservice. To illustrate that you can check with the Syndicai model repository.

Connect a repository

When the model is ready, it's time for deployment. Therefore, the only thing you need to do is log in to Syndicai Platform, go to the Models page and click New Model. Then, fill the form with the repository URL and path to the model (see the picture below).

After pressing Add Model you will see the form that will ask you for the Repository URL, and path.

Later click Add, and you will be redirected to Model Profile.

Create a deployment

Finally, in the Model Profile, click the Deploy button. Please fill the form with the name of the deployment (e.g., Production 1) and the name of the branch (in our case, it's master).

Provide a name and the branch in order to create a new Deployment.

Later confirm the form by clicking Add and the Platform will create the new deployment with the first release. Afterward go to the Releases tab, and click on the recent one #1 to explore logs from the building and starting process.

In the Deployment Profile go to the Releases tab in order to see the history of your deployment.

In order to get the Running status of the deployment waits for a couple of minutes.

Step 3: Integrate

Great! We have our scalable API, so theoretically, the job is done. Now, you can run a model on the Platform or integrate the API with your app.

Edit the box with the sample code in order to test the Deployment on the platform.

In order to test the deployment on the Platform, go to Deployment Profile Overview and scroll down to Validate & Integrate section. Edit the code editor with sample data by pasting the following code.

"url": ""

Later click Update to save and close the box. Click Send request to test the model. Remember that your deployment needs to have Running status to work!

In the further step, you can integrate the API of the model with some React app or a website to build a Showcase.

syndicai showcase pytorch model
Integrate a model with the Syndicai Showcase page.

In order to do it, just fork the repository with the Showcase and explore yolov5 implementation on your own.


To summarize, the main goal of that tutorial was to deploy the PyTorch model in production quickly. No matter your specialty, whether you are a Data Scientist, AI Researcher, or Machine Learning Engineer, currently you can deliver an AI model without any configuration and infrastructure setup.

You can also explore How to deploy Keras model.

* * *

If you found that material helpful, have some comments, or want to share some ideas for the next one - don't hesitate to drop us a line via slack or mail. We would love to hear your feedback!

You might like these

Deploy yolov5 model in a few simple clicks

February 2, 2022
Michał Zmysłowski

Train yolov5. A quick guide from a model to the actual use case.

February 2, 2022
Michał Zmysłowski