How to deploy a PyTorch model in minutes

Marcin Laskowski
deploy pytorch model

In this tutorial, we will cover a simple and fast way to deploy a PyTorch model at scale without a docker, and Kubernetes setup.

I’ve trained my model. What’s next?

Have you ever wanted to deploy a Machine Learning model in order to run it on a website or some mobile app?

In the past, I was exactly in your situation trying different tools, resources. Hundreds of hours spent on building a simple one-page demo app that takes an image as an input and returns one word: Cat. I’ve experienced that the main focus of most Machine Learning Deployment tutorials ends up with building a simple flask app, creating a docker, and setting up the cloud infrastructure. However, it is still time-consuming and difficult. There are many problems with ai model deployment.

Since that time I’ve managed to find a way to solve that problem in just three easy steps.

Let’s have a look and deploy a PyTorch model (Check also How to deploy keras model).

Step 1: Develop

In the first step, we need to have a trained model. For this purpose, we will use a pre-trained PyTorch YoloV5.

For those who don’t know what Yolo is a real-time object detection framework and stands for You Only Look Once, which means that the image passes only once through the Fully Convolutional Neural Network. Comparing to previous versions the V5 is much faster and much smaller (only 27Mb).

Input of the model (on the left) and output (on the right)

In order to run the algorithm locally, you can copy and paste a script from PyTorch Hub. The script will automatically download a pre-trained model and sample images to inference a model.

import cv2
import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).fuse().autoshape()  # for PIL/cv2/np inputs and NMS

# Images
for f in ['zidane.jpg', 'bus.jpg']:  # download 2 images
    print(f'Downloading {f}...')
    torch.hub.download_url_to_file('' + f, f)
img1 ='zidane.jpg')  # PIL image
img2 = cv2.imread('bus.jpg')[:, :, ::-1]  # OpenCV image (BGR to RGB)
imgs = [img1, img2]  # batched list of images

# Inference
results = model(imgs, size=640)  # includes NMS

# Results  # .show() results, .save() jpgs, or .print() to screen

Step 2: Deploy

We have our basic script with the model, so now we can deploy it in the cloud.

For this purpose, we will use the Syndicai Platform which helps developers deliver AI in production effortlessly without setting up an infrastructure. No config needed, no deep tech knowledge required. Syndicai takes care of security, scalability, monitoring, and more. You just need to prepare a model and connect your git repository. The model will be deployed automatically.

AI Model Deployment: Traditional Approach vs Syndicai

Prepare a model

During the model preparation process you need to ensure the following files are in the git repository:

1st file: requirements.txt – file with all libraries and frameworks needed to recreate model’s environment


2nd file: – main file with the PythonPredictor python class responsible for model prediction.

import torch
from PIL import Image 
from helpers import draw_box, url_to_img, img_to_bytes

class PythonPredictor:

    def __init__(self, config):
        """ Download pretrained model. """
        self.model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()

    def predict(self, payload):
        """ Run a model based on url input. """

        # Inference
        img = url_to_img(payload["url"])
        results = self.model(img)

        # Draw boxes
        boxes = results.xyxy[0].numpy()
        box_img = draw_box(img, boxes)

        # Save image"sample_data/output.png", "PNG")

        return img_to_bytes(box_img)

Those files are necessary to correctly recreate your environment, build the infrastructure, and finally run the model in the form of a webservice. You can check with the Syndicai model repository.

Connect a repository

When the model is ready it’s time for deployment. The only thing that you need to do at this stage is to log in to Syndicai Platform, click New Model on the Overview page, and follow the steps in the form. After the model is deployed the whole magic happens in order to give you the model API in the Integrate section.

Deploy a Model by connecting the repository.

For more information about the model preparation or deployment process go to Syndicai Docs.

Step 3: Integrate


We have our scalable API so theoretically the job is done. You can run a model on the Platform or integrate the API with your app.

In order to make an inference the model go to Syndicai platform and paste a sample input script in the Run a model section

    "url": ""

Remember that your model needs to be Active in order to work!

Run a model on the Platform

API of the model can be integrated with some React app or a website in order to build a Showcase.

Integrate a model with Syndicai Showcase page.

Fork the repository with the Showcase and explore yolov5 implementation on your own.


The main goal of that tutorial was to easily deploy the PyTorch model in production. No matter what is your specialty, whether you are a Data Scientist, a Machine Learning Engineer, Backend Engineer, or DevOps. Now you are able to deliver an AI model without any configuration and infrastructure setup.

You can also explore How to deploy Keras model.

If you found that useful, or you want to get more of those types of tutorials – please drop us a line by mail or catch us on slack.