A quick guide on how to deploy depth estimator at scale.

A quick guide on how to deploy a face blurring at scale.


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/fastpi_cloudrun.png" alt="deploy ml model using fastapi" class="wp-image-602" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/fastpi_cloudrun.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/02/fastpi_cloudrun-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/fastpi_cloudrun-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">A step-by-step tutorial on how to deploy a PyTorch implementation of the AlexNet Image Classifier using FastAPI, Docker, and Cloud Run.</p>



<h2>Introduction</h2>



<p>Welcome to another tutorial that aims to show you how to bring any trained AI model to live by deploying them into production.</p>



<p>In the previous tutorial, <a href="https://data.syndicai.co/blog/deploy-ml-model-with-flask/">deploy ml using flask</a>, we showed you a step-by-step guide of how to deploy a machine learning model using flask, docker, and cloud run. Therefore, the main goal of the following article we will go a step further and show you a better way.</p>



<p>This time we will deploy ML with FastAPI, specifically, an <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/pytorch/image_classifier" target="_blank" rel="noreferrer noopener nofollow">AlexNet Image Classifier model</a>. </p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/alexnet.png" alt="AlexNet model" class="wp-image-608" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/alexnet.png 1007w, https://data.syndicai.co/wp-content/uploads/2021/02/alexnet-300x156.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/alexnet-768x400.png 768w" sizes="(max-width: 1007px) 100vw, 1007px" /><figcaption>AlexNet model takes image as input and output the predicted class</figcaption></figure>



<p>The whole workflow is pretty straightforward. First, we will wrap a model into a web service, later create a Dockerfile, and finally host a service on GCP.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-1024x261.png" alt="deploy ml model using fastapi" class="wp-image-603" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-1024x261.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-300x76.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-768x196.png 768w, https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-1536x391.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-2048x522.png 2048w, https://data.syndicai.co/wp-content/uploads/2021/02/fastapi-1568x400.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Machine Learning model deployment workflow</figcaption></figure>



<p>The final version of the code for this tutorial can be found <a href="https://github.com/mlaskowski17/ml-fastapi-docker" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">here</a>.</p>



<h2>Prerequisites</h2>



<p>Before we deploy ML using FastAPI prepare install following requirements:</p>



<ol><li>Basic understanding of Python, Docker, REST</li><li>Access to Google Cloud Platform (if you want to host a model)</li><li>Trained model from <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/pytorch/image_classifier" target="_blank" rel="noreferrer noopener nofollow">the repository</a>.</li><li>Installed <a aria-label="undefined (opens in a new tab)" href="https://docs.docker.com/engine/install/ubuntu/" target="_blank" rel="noreferrer noopener nofollow">docker</a>, <a aria-label="undefined (opens in a new tab)" href="https://docs.docker.com/compose/install/" target="_blank" rel="noreferrer noopener nofollow">docker-compose</a>, <a aria-label="undefined (opens in a new tab)" href="https://cloud.google.com/sdk/docs/install" target="_blank" rel="noreferrer noopener nofollow">gcloud</a>, <a aria-label="undefined (opens in a new tab)" href="https://docs.python-guide.org/starting/install3/linux/" target="_blank" rel="noreferrer noopener nofollow">python 3.6</a></li><li>Installed the following python dependencies: <code>torch==1.7.1 torchvision==0.8.2 fastapi==0.63.0 uvicorn==0.13.3 <code>requests==2.18.4</code></code></li></ol>



<p class="infoblock">💡 <strong>Explore</strong><br>If you are interested in AI model deployment, please check out also a tutorial on how to <a href="https://data.syndicai.co/blog/deploy-yolov5/">deploy yolov5</a> model.</p>



<h2>Expose as a Webservice</h2>



<p>In the first step, we need to create a webservice that will handle all requests.</p>



<p>For this purpose, we will use <strong>FastAPI</strong>, which is a web python framework that is high-performance, easy to learn, fast to code, and ready for high-end production.</p>



<p>What I personally love about the FastAPI is that it&#8217;s built on Starlette &amp; Pydantic, and despite that fact the syntax is very similar to the flask so that you can easily refactor your code to a new framework.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/flask_fastapi-1024x437.png" alt="flask vs fastapi" class="wp-image-604" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/flask_fastapi-1024x437.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/02/flask_fastapi-300x128.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/flask_fastapi-768x328.png 768w, https://data.syndicai.co/wp-content/uploads/2021/02/flask_fastapi.png 1200w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Code comparison between flask (left) and fastAPI (right)</figcaption></figure>



<h3>Download a model</h3>



<p>As a first step download (or copy) a repository with <a href="https://github.com/syndicai/models/tree/master/pytorch/image_classifier">AlexNet Image Classifier</a> that was written in the PyTorch framework. It takes an image in the form of a URL as an input and returns a class of the picture.</p>



<p>The structure of the repository should look as follows:</p>



<pre class="wp-block-code bash"><code>├── README.md
├── requirements.txt
├── sample_data
│   ├── input.jpg
│   └── sample_input.json
└── syndicai.py</code></pre>



<h3>Create an app.py</h3>



<p>In the main directory create a new file called <code>app.py</code> and paste the code below:</p>



<pre class="wp-block-code python"><code>import json
import uvicorn
from fastapi import FastAPI
from syndicai import PythonPredictor

app = FastAPI()


@app.get("/")
def hello():
    """ Main page of the app. """
    return "Hello World!"


@app.get("/predict")
async def predict(url: str):
    """ Return JSON serializable output from the model """
    payload = {'url': url}
    classifier = PythonPredictor("")
    return classifier.predict(payload)

if __name__ == '__main__':
    uvicorn.run(app, host='0.0.0.0', port=8000)</code></pre>



<p>You can see that at the beginning of the file, we import important <code>json</code>, <code>uvicorn</code>, <code>FastAPI</code> as well as a <code>PythonPredictor</code> class form the <em>syndicai.py</em> file.</p>



<p>Next, we create an instance of this class. The first argument is the name of the application’s module or package. This is needed so that Flask knows where to look for templates, static files, and so on (For more information have a look at the <a aria-label="undefined (opens in a new tab)" href="https://fastapi.tiangolo.com/" target="_blank" rel="noreferrer noopener">FastAPI documentation</a>).</p>



<p>Later, we have created two HTTP requests in the <em>paths</em> <code>/</code> and <code>/predict</code>. Both <em>paths</em> take <code>GET</code> <em>operations</em> (also known as HTTP <em>methods</em>)</p>



<h3>Run a service</h3>



<p>In order to run a webservice use <code>python app.py</code> command and paste <code>http://localhost:8000</code> in the web browser to see <em>&#8220;Hello World!&#8221;</em>.</p>



<p>What is great about FastAPI is that it <strong>generates the documentation</strong> on the go when you are developing the API which is the most requested thing from all the developers. Just run <code>http://localhost:8000/docs</code> or <code>http://localhost:8000/redoc</code> in the browser.</p>



<p>In order to run a model paste <code>https://i.imgur.com/PzXprwl.jpg</code> in the parameter section of the <code>/predict</code> endpoint. </p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/docss-1024x511.png" alt="FastAPI docs " class="wp-image-607" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/docss-1024x511.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/02/docss-300x150.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/docss-768x384.png 768w, https://data.syndicai.co/wp-content/uploads/2021/02/docss-1536x767.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/02/docss-1568x783.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/02/docss.png 1904w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Run a model using Swagger generated by FastAPI: http://localhost:8000/docs</figcaption></figure>



<p>Keep in mind the first model run will take some time because the model will download missing files (weights)!</p>



<p>Congrats! You have just created a FastAPI app!</p>



<p class="infoblock">💡 <strong>Explore</strong><br>Learn more about Web services for ML. <br><a aria-label="undefined (opens in a new tab)" href="https://www.analyticsvidhya.com/blog/2020/11/fastapi-the-right-replacement-for-flask/" target="_blank" rel="noreferrer noopener nofollow">FastAPI: The right replacement for Flask?</a><br><a aria-label="undefined (opens in a new tab)" href="https://github.com/ycd/manage-fastapi" target="_blank" rel="noreferrer noopener nofollow">FastAPI new project boilerplate</a><br><a href="https://www.udemy.com/course/python-api-development/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Complete Backend (API) Development course</a></p>



<h2>Wrap with a Docker</h2>



<p>In the next, we need to take our service with all dependencies and wrap it using docker so that it will be reproducible.</p>



<p>For this purpose, we will use <strong>Docker</strong> which is an open-source application to create, manage, deploy, and replicate applications using containers.</p>



<p><strong>Containers</strong> can be thought of as a package that houses dependencies required by the app to run at an operating system level. This means that each application deployed using Docker lives in an environment of its own and its requirements are handled separately.</p>



<p>When deploying a FastAPI app with Docker allows us to replicate the application across different servers with no reconfiguration.</p>



<h3>Create a Dockerfile</h3>



<p>In order to use docker, you need to first create a <em><strong>Dockerfile</strong></em> in the main directory with the following code.</p>



<pre class="wp-block-code docker"><code># Use python as base image
FROM python:3.6-stretch

# Use working directory /app/model
WORKDIR /app/model

# Copy and install required packages
COPY requirements.txt .
RUN pip install --trusted-host pypi.python.org -r requirements.txt

# Copy all the content of current directory to working directory
COPY . .

# Set env variables for Cloud Run
ENV PORT 80
ENV HOST 0.0.0.0

EXPOSE 80:80

# Run flask app
CMD &#91;"uvicorn", "app:app", "--host", "0.0.0.0", "--port", "80"]</code></pre>



<p><strong>The Dockerfile</strong> is a text document that contains commands used to assemble the image. Therefore, just paste the code below inside. Creating a Dockerfile it&#8217;s worth to have in mind a couple of best practices:</p>



<ol><li>Layer with <em>requirements.txt</em> installation needs to be above the layer where you copy model files. Thank to that approach, each time when you will change something in your model, docker will rebuild the image without installing dependencies again.</li><li>Use more specific tags and dependency versions. It will prevent you from failure when new library updates are released.</li><li>Change the working directory for your files, don&#8217;t work in the root.</li></ol>



<p>For more detailed information you can read the <a aria-label="undefined (opens in a new tab)" href="https://docs.docker.com/engine/reference/builder/" target="_blank" rel="noreferrer noopener nofollow">Dockerfile reference</a>.</p>



<h3>Create a docker-compose</h3>



<p>Additionally to <em>Dockerfile</em> let&#8217;s create a <em>docker-compose.yaml</em> that will help us test the app locally.</p>



<pre class="wp-block-code"><code>version: '3'
services:

  flask_classifier:
    build: .
    ports:
      - 80:80</code></pre>



<p><strong>The docker-compose</strong> is a tool for defining and running multi-container Docker applications using <em>yaml</em> file. Compose works in all environments: production, staging, development, testing, as well as CI workflows and it allows very quickly run a dockerized app.</p>



<p>In order to check if the app is working properly inside the docker just run <code>docker-compose up --build</code> in the terminal. The service should be available under the <code>localhost</code> (port 80).</p>



<p class="infoblock">💡 <strong>Explore</strong><br>Please check out the following resources to learn more about Docker.<br><a href="https://docs.docker.com/engine/reference/builder/" target="_blank" rel="noreferrer noopener">Dockerfile reference</a><br><a href="https://phoenixnap.com/kb/docker-image-vs-container" target="_blank" rel="noreferrer noopener">Docker Image vs Container</a><br><a href="https://www.docker.com/blog/intro-guide-to-dockerfile-best-practices/" target="_blank" rel="noreferrer noopener">Intro guide to Dockerfile best practices</a></p>



<h2>Deploy on Cloud Run</h2>



<p>At this stage, the repository structure should look as follows.</p>



<pre class="wp-block-code bash"><code>.
├── app.py
├── docker-compose.yaml
├── Dockerfile
├── README.md
├── requirements.txt
├── sample_data
│   ├── input.jpg
│   └── sample_input.json
└── syndicai.py</code></pre>



<p>Previously we exposed our app in the form of a web service and created a Dockerfile that allows us to customize the runtime of our container. The last step of this tutorial is service deployment so that it will be out to the world. Therefore, we will use a <strong>Cloud Run</strong>, a serverless platform for hosting docker containers. It is fully manageable so you don&#8217;t need to worry about the infrastructure.</p>



<p>In order to deploy a service, you need to run two commands. In addition, remember to substitute PROJECT_ID with the name of your GCP project!</p>



<pre class="wp-block-code bash"><code># Build a docker image 
gcloud builds submit --tag gcr.io/PROJECT_ID/flask-classifier

# deploy a docker image to cloud run
gcloud run deploy --image gcr.io/PROJECT_ID/flask-classifier \
  --platform managed \
  --port 80 \
  --memory 1G</code></pre>



<p>You can see the deployed service in the Cloud Run.</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-1024x523.png" alt="cloud run syndicai " class="wp-image-510" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-1024x523.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-300x153.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-768x393.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1.png 1485w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Cloud Run dashboard with deployed model</figcaption></figure></div>



<p>In order to run your model, just copy the url of the service and use the following request.</p>



<pre class="wp-block-code"><code>curl --request GET \
  --url 'https://CLOUD_RUN_SERVICE_URL/predict?url=https://i.imgur.com/PzXprwl.jpg'</code></pre>



<p>Wow! You have just deployed the Image Classifier, Tutorial <em>Deploy ML using FastAPI </em>completed!</p>



<div class="wp-block-image justify-center"><figure class="aligncenter size-large"><img src="https://media.giphy.com/media/PkXrOxe77MbmavlfWa/giphy.gif" alt="fastapi mlops syndicai"/></figure></div>



<h2>Conclusion</h2>



<p>To sum up, in the following tutorial you had a chance to learn how to deploy a model using FastAPI, docker and Cloud Run.</p>



<p>As far as FastAPI is concerned I would highly recommend using it when deploying a model. In short, it is very easy to set up, works in async, and automatically generates docs.</p>



<p class="text-center">* * *</p>



<p>If you found that tutorial helpful, or have some questions, please let us know via <a href="mailto:hello@syndicai.co" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">mail</a> or join us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">slack</a>. We would love to hear your feedback.</p>



<p>We would also like to inspire you via our <a aria-label="undefined (opens in a new tab)" href="https://showcase.syndicai.co/" target="_blank" rel="noreferrer noopener">AI models Showcase</a> page and give you a warm invitation to try out our <a href="https://syndicai.co/">MLOps platform</a> that significantly speeds up the work of AI Teams.</p>


Deploy ML with FastAPI, Docker, and Cloud Run


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/start_with_mlops.png" alt="start with mlops" class="wp-image-629" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/start_with_mlops.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/02/start_with_mlops-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/start_with_mlops-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">How to start with MLOps, what it is exactly, and how to take it into account when creating an AI strategy with your AI team.</p>



<h2>Introduction</h2>



<p>According to the latest Tech Trends report introduced by Deloitte, companies will need to rethink AI adoption in 2021 taking MLOps into consideration.</p>



<p>Why is that?</p>



<p>More and more companies try to apply AI in their products, processes, or services. However, many of them face <a href="https://data.syndicai.co/blog/problems-with-ai-model-deployment/">problems with AI model deployment</a>. As a result, they need to finish the project with just a proof of concept without going into production.</p>



<p>According to the IDC report, about 28% of machine learning projects do not succeed. The problem is a lack of necessary expertise, production-ready data, and integrated development environments. While roughly 47% fail to even make it out of the experimental phase and into production.</p>



<p>The solution to that problem is applying MLOps that aims to help with AI delivery and management in production environment.</p>



<p>Therefore, in the following article, we will discuss how to start with MLOps. What it is exactly, and how to take it into account when creating an AI strategy.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/deloitte-1024x399.png" alt="MLOps in Tech Trends 2021" class="wp-image-631" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/deloitte-1024x399.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte-300x117.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte-768x299.png 768w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte-1536x599.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte-1568x611.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte.png 1901w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>MLOps: Trend 4 of 9 from Tech Trends 2021 report</figcaption></figure>



<h2>What is MLOps?</h2>



<p><a href="https://data.syndicai.co/blog/machine-learning-operations/">Machine Learning Operations</a> (MLOps) is a set of best practices that aim to seamlessly deploy, integrate, and run machine learning models in a production environment. Unlike, DevOps it is not only based on continuous integrations and continuous deployment but also continuous training.</p>



<blockquote class="wp-block-quote"><p>&#8220;Through continuous development, testing, deployment, monitoring, and retraining, MLOps can improve collaboration among teams and shorten development life cycles, thereby enabling faster, more reliable, and more efficient model deployment, operations, and maintenance as well.&#8221;</p><cite>Deloitte : : Tech Trends 2021</cite></blockquote>



<p>When applying MLOps you own the control over scalability, monitoring, maintenance, and governance of the whole AI model pipeline. Therefore, the entire model delivery process is fast and is not at risk of failure.</p>



<p>In the following paragraphs, we will try to investigate how to apply MLOps in your organization.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2-1024x575.png" alt="MLOps: Industrialized AI" class="wp-image-635" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2-1024x575.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2-300x168.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2-768x431.png 768w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2-1536x862.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2-1568x880.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/02/deloitte2.png 1596w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Scaling model development and operations with a dose of engineering and operational discipline</figcaption></figure>



<h2>Establish your ML process maturity</h2>



<p>In the first step, we need to establish an MLOps process maturity of the organization. According to Google, there are three levels of ML process maturity in the company. Each of them depends on the total percentage of automated tasks in the entire ML workflow.</p>



<h3>Level 0: Manual work</h3>



<p>At the first level, the whole process of AI model building and deploying is entirely manual. It is characterized by a large gap between the Data Science team and the operational team. There are no CI / CD pipes and no performance monitoring.</p>



<p>Level 0 is sufficient only when models are rarely deployed to production. </p>



<h3>Level 1: Partially automated pipeline</h3>



<p>The next level consists of at least one automated pipeline that allows Data Scientists to speed up repeatable tasks. That might be for instance Continuous training of the model in production. The model is automatically trained based on new incoming data initialized by some trigger.</p>



<p>This level occurs very often in small AI teams, that need to handle and adapt to a fast-changing ML environment.</p>



<h3>Level 2: Fully automated pipeline</h3>



<p>The last level is characterized by a fully automated CI/CD pipeline. Data Scientists are able to rapidly explore new hypotheses and deploy them into the production environment to validate assumptions.</p>



<p>Such a level of maturity consists of data, model, and code versioning, monitoring. In addition, it is highly secure and easily manageable and consists of Continuous integration, deployment, and training pipelines.</p>



<p>Summing up there are three levels of MLOps maturity, and each of them is characterized by different needs and challenges.</p>



<p>Level 0 is the most common, especially among companies that start to build in-house AI teams. Therefore, in the next paragraphs, we will focus on the path from level 0 to level 1.</p>



<h2>How to go from zero to one?</h2>



<p>However, there are three things that significantly speed up work and helps companies go from level zero to one. Let&#8217;s have a look at them in details.</p>



<h3>Experiment Tracking</h3>



<p>When talking about AI workflow (MLOps workflow) it is important to mention that there are two main phases. The first one, the experimental, when we explore data, build features, and train models. The second one, operational, when we actually deploy model to production and deal with all operational things.</p>



<p>The most important step towards Level 1 is to take care of that first phase. Tracking all experiments will allow Data Scientists to reproduce obtained in the past results very easily. It is important to emphasize that in order to get a good <em>reproducibility</em> you need to take care of not only model versioning but also code and data versioning.</p>



<p>Keeping track of all your work and being able to recreate that work when needed is a crucial thing in the Data Science world.</p>



<h3>Continuous Deployment</h3>



<p>Second very important improvement that is worth to take into consideration when going from level <em>zero to one</em> is Continuous Deployment.</p>



<p>Not long ago I spoke with a couple of SME&#8217;s (Small-Medium size Enterprises) and it occurs that there is no problem with the Data Science workflow. However, there is practically no operational and management layer when it comes to production models.</p>



<p>Why is that the problem? &#8230;because models that have 99% accuracy, but are not deployed does not give any business value.</p>



<p>The solution is to build some abstraction operational layer for AI model deployments. It would allow Data Scientists to easily deliver models in the form of microservices effortlessly.</p>



<h3>Performance Monitoring</h3>



<p>If we managed to deploy models to production it is important to know whether they are alive. Therefore, adding some monitoring system that give us basic information about resource consumption is the way to go.</p>



<h2>Should I buy or build?</h2>



<p>We know &#8220;where&#8221; we are and &#8220;what&#8221; we want to achieve, so the question that arises currently on the horizon is &#8220;How?&#8221;.</p>



<p>There are two solutions. You can either build your own (in-house) system that will help you speed up your work, or you can buy one. There are pros and cons when it comes to both approaches.</p>



<p>I love <a aria-label="undefined (opens in a new tab)" href="https://cnvrg.io/build-vs-buy-data-science-platform/" target="_blank" rel="noreferrer noopener nofollow">the info-graphics</a> introduced by one of the best companies in space of AI that exactly hit the spot when it comes to buy vs build perplexity. </p>



<p>However, there are some use-cases that need a special approach when it comes to AI delivery, and the answer is not so obvious. For instance, Energy supply sector deals with specific requirements, and buying an industry agnostic platform very often is not a solution at all.</p>



<p>It is best for you to estimate the costs, benefits of its option and make the decision yourself&#8230;</p>



<p>&#8230;and chose Syndicai! 😀</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/02/buybuild-1024x426.png" alt="buy or build ai platform" class="wp-image-637" srcset="https://data.syndicai.co/wp-content/uploads/2021/02/buybuild-1024x426.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/02/buybuild-300x125.png 300w, https://data.syndicai.co/wp-content/uploads/2021/02/buybuild-768x320.png 768w, https://data.syndicai.co/wp-content/uploads/2021/02/buybuild.png 1362w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Challenges of building an ML management platform according to Algorithmia</figcaption></figure>



<h2>Conclusion</h2>



<p>The main purpose of this article was to build awareness that proper collection and cleaning of data and thorough building and training of the model will not ensure success in your ML project. You need to remember about basics of MLOps, thinking about AI delivery when models trained.</p>



<p class="text-center">* * *</p>



<p>If you found that content helpful, or have some questions related to that topic, please let us know via <a aria-label="undefined (opens in a new tab)" href="mailto:hello@syndicai.co" target="_blank" rel="noreferrer noopener nofollow">mail</a> or join us on <a aria-label="undefined (opens in a new tab)" href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" rel="noreferrer noopener nofollow">slack</a>. We would love to hear your feedback.</p>



<p>We would also like to inspire you via our <a aria-label="undefined (opens in a new tab)" href="https://showcase.syndicai.co/" target="_blank" rel="noreferrer noopener">AI models Showcase</a> page and give you a warm invitation to try out our <a href="https://syndicai.co/">MLOps platform</a> that significantly speeds up the work of AI Teams.</p>


How to start with MLOps?


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/ddi_deoldify.png" alt="Colorize black and white image with syndicai" class="wp-image-543" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/ddi_deoldify.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/01/ddi_deoldify-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/ddi_deoldify-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Deploy a DeOldify model at scale in a few simple clicks without setting up a webservice, Docker, Kubernetes, Google Cloud or AWS.</p>



<h2>Introduction</h2>



<p>Did you know that in the early 90s photographers and artists, in an attempt to create more realistic images, started hand-color monochrome photographs? Did you know that the first person who tried that technique was Johann Baptist Isenring who produced colored Daguerreotypes, using a mixture of Arabic gum and pigments? Did you know that the best-selling hand-colored photographer of all time was Wallace Nutting &#8211; England minister? </p>



<p>These are just a few interesting facts that confirm that the history of hand-coloring was very lush and that the whole technique is more like art and less like a service. However, things changed with the advent of artificial intelligence.</p>



<p>Deep learning algorithms far outperform the old methods (photo below) as they were able not only to color the images but also restore them (in minutes!).</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1-1024x508.png" alt="charlie chaplin colorize by deoldify deployed on syndicai" class="wp-image-544" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1-1024x508.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1-300x149.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1-768x381.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1-1536x762.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1-1568x778.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_1.png 1800w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Charlie Chaplin as Shahdov from &#8220;A King in New York&#8221; (1957).</figcaption></figure>



<p>That is why, inspired by such amazing results, we decided to show you how to become an artist yourself. In this tutorial, we want to show you how to implement one of the top photo colorizing algorithm, so that you can later create a web application or demo showcase from it.</p>



<p>Let&#8217;s start!</p>



<p class="infoblock">💡 <strong>Explore</strong><br>If you are interested in AI model deployment, please check out also a tutorial on how to <a href="https://data.syndicai.co/blog/deploy-yolov5/">deploy yolov5</a> model.</p>



<h2>Step 1: Develop a DeOldify model</h2>



<p>In order to deploy a model, we need to go through three quick steps.</p>



<p>In the first step, we need to train a model, so that inference will be possible. For simplicity, we will use a pretrained version of the DeOldify model introduced in 2018. It is the state of the art algorithm to colorize black &amp; white images that combines Self-Attention GANs and Two Time-Scale Update Rule. </p>



<p>You can see the model input image and output below, and the results are astonishing!</p>



<figure class="wp-block-image size-large justify-center"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_2.png" alt="gilling castle interior colorize by deoldify deployed on syndicai
" class="wp-image-545" title="" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_2.png 990w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_2-300x182.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_2-768x465.png 768w" sizes="(max-width: 990px) 100vw, 990px" /><figcaption>Black and white image (on the left) parsed through the DeOldify colorize image (on the right). In the picture, we can see Gilling Castle interior (1909)</figcaption></figure>



<p>When the model is trained it needs to be uploaded to the GitHub repository. We have already prepared one for you under the following <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/fastai/deoldify" target="_blank" rel="noreferrer noopener nofollow">url</a>, so you can just fork it and play with the model.</p>



<p>Weights of the trained model are ready and the code is already on GitHub, so we can jump to the next step.</p>



<h2>Step 2: Deploy a model</h2>



<p>In the next step we will deploy a model. </p>



<p>When deploying a model you have a couple of options on how to do it properly. In many situations, that decision is highly influenced by the particular use-case. For instance, in our case we just want to perform the simple inference on cloud. The most popular way to do it is by creating a webservice, building a docker, and later setting up the infrastructure on cloud. However, such an approach has a lot of disadvantages like difficult resource management, no versioning, or lack of basic security. Therefore, we will use the solution that significantly simplifies and speed up the whole process &#8211; a <a href="/">Syndicai</a> Platform.</p>



<p class="infoblock">💡 <strong>Explore</strong><br>If you are interested in the traditional AI model deployment, please go to <a href="https://data.syndicai.co/blog/deploy-ml-model-with-flask/">deploy ML with flask</a> article.</p>



<h3>Prepare a model</h3>



<p>Apart from putting your model in the GitHub repository, you have to upload two additional files (<code>syndicai.py</code>and <code>requirements.txt</code>) in the main directory of your model.</p>



<p>The first file, <code>syndicai.py</code>, is the main file that consists of <code>PythonPredictor</code> python class responsible for model prediction. In this case, the models take an URL of the black-and-white image as input and output colorized image in the form of base64.</p>



<pre class="wp-block-code python"><code># This must be the first call in order to work properly!
import io
import base64
import urllib.request
from deoldify import device
from deoldify.device_id import DeviceId
device.set(device=DeviceId.GPU0)

import torch

if not torch.cuda.is_available():
    print('GPU not available.')

import fastai
from deoldify.visualize import *
import warnings
warnings.filterwarnings("ignore", category=UserWarning, message=".*?Your .*? set is empty.*?")


url = "https://data.deepai.org/deoldify/ColorizeArtistic_gen.pth"
model_path = "./models/ColorizeArtistic_gen.pth"


class PythonPredictor:
    
    def __init__(self, config):
        if not os.path.exists("models/ColorizeArtistic_gen.pth"):
            urllib.request.urlretrieve(url, model_path)
        self.colorizer = get_image_colorizer(artistic=True)
    
    def predict(self, payload):
        
        render_factor = 19  #@param {type: "slider", min: 7, max: 40}
        
        img = self.colorizer.plot_transformed_image_from_url(url=payload&#91;"url"],
                                                         render_factor=render_factor)
        
        im_file = io.BytesIO()
        img.save(im_file, format="PNG")
        im_bytes = base64.b64encode(im_file.getvalue()).decode("utf-8") 

        return im_bytes</code></pre>



<p>The second file required by the Syndicai platform is needed to recreate the model environment. It consists of a list of libraries and their versions.</p>



<pre class="wp-block-code text"><code>fastai==1.0.51
wandb
tensorboardX==1.6
ffmpeg
ffmpeg-python==0.1.17
youtube-dl>=2019.4.17
jupyterlab
opencv-python>=3.3.0.10
pillow==6.2.2</code></pre>



<h3>Connect the repository to Syndicai</h3>



<p>We are halfway through the journey of our model deployment. Now you will deploy a model by connecting a repo to the Syndicai platform.</p>



<p>In order to connect your repo just go to <a href="https://app.syndicai.co/">https://app.syndicai.co/</a>, login, click <strong>New Model</strong> on the <strong>Overview</strong> page, and follow the steps in the form. As soon as you finish, you will notice that the infrastructure will start building. Therefore, you will have to wait a few minutes for the model to become <em>Active</em>.</p>



<figure class="wp-block-image size-large justify-center"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/connect-1024x514.png" alt="" class="wp-image-556" title="" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/connect-1024x514.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/connect-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/connect-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/connect-1536x772.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/connect-1568x788.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/connect.png 1905w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Syndicai Platform let you deploy a model just via Connecting a repo.</figcaption></figure>



<p>Seriously that&#8217;s all?</p>



<p>Yes, indeed. Syndicai creates a webservice, package a model and host it in a serverless way so that you don&#8217;t have to take care of scalability and model management.</p>



<p class="infoblock">📚 <strong>Learn</strong><br>For more information about the model preparation or deployment process go to <a href="https://docs.syndic.ai/">Syndicai Docs</a>.</p>



<h2>Step 3: Integrate</h2>



<p>Congrats!</p>



<p>Your model is deployed so now you can access it using REST API either via Platform or via terminal. </p>



<div class="wp-block-image justify-center"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/run-1024x515.png" alt="Syndicai platform deploy colorizer" class="wp-image-557" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/run-1024x515.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/run-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/run-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/run-1536x773.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/run-1568x789.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/run.png 1904w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Deployed model with status Active</figcaption></figure></div>



<p>In order to test it out quickly on the Platform, go to the model <strong>Overview</strong> page and paste the <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/blob/master/fastai/deoldify/sample_data/sample_input.json" target="_blank" rel="noreferrer noopener nofollow">sample input</a> script in the <strong>Run a model</strong> section.</p>



<pre class="wp-block-code bash"><code>{
    "url": "https://i0.wp.com/www.brainpickings.org/wp-content/uploads/2013/05/einstein11.jpg"
}</code></pre>



<p>Remember that your model has to be <em>Active</em> in order to work!</p>



<p>After that, If everything works fine, you can go to the model <strong>Integrate</strong> page and create the showcase demo. For example, we have created a <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/showcase" target="_blank" rel="noreferrer noopener nofollow">Colorizer</a> &#8211; a react web app that helps you interact with your model in w very simple way. You just paste the url of the image and hit the <strong>Run Model</strong> button! The whole code is <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/showcase" target="_blank" rel="noreferrer noopener nofollow">open-sourced</a> so you can fork the repo and play with it.</p>



<figure class="wp-block-image size-large justify-center"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/showcase-1024x515.png" alt="Colorizer - a demo showcase for DeOldify model
" class="wp-image-552" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/showcase-1024x515.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/showcase-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/showcase-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/showcase-1536x772.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/showcase-1568x789.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/showcase.png 1901w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Colorizer is a demo showcase for DeOldify model</figcaption></figure>



<h2>Summary</h2>



<figure class="wp-block-image size-large justify-center"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3-1024x641.png" alt="marilyn monroe colorize by deoldify deployed on syndicai" class="wp-image-546" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3-1024x641.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3-300x188.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3-768x481.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3-1536x961.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3-1568x981.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/deoldify_3.png 1636w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Marilyn Monroe posing toward the photographer</figcaption></figure>



<p>In conclusion, the following tutorial lets you explore the fast and easy way to deploy DeOldify model. You could notice that creating a webserivce, building a docker, and setting up the infrastructure is not necessary to deliver a model in production.</p>



<p class="text-center">* * *</p>



<p>If you found that tutorial helpful, or have some questions, please let us know via <a href="mailto:hello@syndicai.co" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">mail</a> or join us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">slack</a>. We would love to hear your feedback.</p>



<p>We would also like to inspire you via our <a aria-label="undefined (opens in a new tab)" href="https://showcase.syndicai.co/" target="_blank" rel="noreferrer noopener">AI models Showcase</a> page and give you a warm invitation to try out our <a href="https://syndicai.co/">MLOps platform</a> that significantly speeds up the work of AI Teams.</p>


How to deploy DeOldify in minutes?


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/flask_cloudrun.png" alt="flask, docker and cloud run" class="wp-image-509" title="flask cloud run mlops" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/flask_cloudrun.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/01/flask_cloudrun-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/flask_cloudrun-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">A step-by-step tutorial on how to deploy a PyTorch implementation of the Alexnet Image Classifier using Flask, Docker, and Cloud Run.</p>



<h2>Introduction</h2>



<p>Once you have a working model, the next step is to put it to production!</p>



<p>You have many options on how to do it. It mostly depends on the use-case which one to choose. If you want to explore all of them, please check out the article about AI model deployment.</p>



<p><strong>In this article</strong>, we will deploy ML model using Flask. Specifically, an <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/pytorch/image_classifier" rel="noreferrer noopener nofollow" target="_blank">AlexNet Image Classifier</a> using THE most popular stack: Flask, Docker, and Cloud Run. The whole workflow is pretty straightforward. Wrap a model into a web service, create a Dockerfile, and host on GCP. The final version of the code for this tutorial can be found <a aria-label="undefined (opens in a new tab)" href="https://github.com/mlaskowski17/ml-flask-docker" target="_blank" rel="noreferrer noopener nofollow">here</a>. </p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/alexnet.png" alt="" class="wp-image-520" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/alexnet.png 1007w, https://data.syndicai.co/wp-content/uploads/2021/01/alexnet-300x156.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/alexnet-768x400.png 768w" sizes="(max-width: 1007px) 100vw, 1007px" /></figure></div>



<p>Easy peasy, so let&#8217;s start!</p>



<h2>Prerequisites</h2>



<p>This tutorial requires the following:</p>



<ol><li>Basic understanding of Python, Docker, REST</li><li>Access to Google Cloud Platform (if you want to host a model)</li><li>Trained model from <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/pytorch/image_classifier" target="_blank" rel="noreferrer noopener nofollow">the repository</a>.</li><li>Installed <a aria-label="undefined (opens in a new tab)" href="https://docs.docker.com/engine/install/ubuntu/" target="_blank" rel="noreferrer noopener nofollow">docker</a>, <a aria-label="undefined (opens in a new tab)" href="https://docs.docker.com/compose/install/" target="_blank" rel="noreferrer noopener nofollow">docker-compose</a>, <a aria-label="undefined (opens in a new tab)" href="https://cloud.google.com/sdk/docs/install" target="_blank" rel="noreferrer noopener nofollow">gcloud</a>, <a aria-label="undefined (opens in a new tab)" href="https://docs.python-guide.org/starting/install3/linux/" target="_blank" rel="noreferrer noopener nofollow">python 3.6</a></li><li>Installed following python dependencies: <code>torch==1.7.1 torchvision==0.8.2 flask==1.1.2 requests==2.18.4</code></li></ol>



<p class="infoblock">💡 <strong>Explore</strong><br>If you are interested in AI model deployment, please check out also tutorial on how to <a href="https://data.syndicai.co/blog/deploy-yolov5/">deploy yolov5</a> model.</p>



<h2>Expose as a Webservice</h2>



<p>Once you have a trained model, you need to create a web service able to handle requests.</p>



<p>For this purpose, we will use <strong>Flask</strong>, a lightweight WSGI web application framework. It provides various tools and libraries for building web apps making the core simple but extensible.</p>



<p>As a first step download (or copy) a repository with <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/pytorch/image_classifier" target="_blank" rel="noreferrer noopener nofollow">AlexNet Image Classifier</a> that was written in the PyTorch framework. It takes an image in the form of a URL as an input and returns a class of the picture.</p>



<p>The structure of the repository should look as follows:</p>



<pre class="wp-block-code bash"><code>├── README.md
├── requirements.txt
├── sample_data
│   ├── input.jpg
│   └── sample_input.json
└── syndicai.py</code></pre>



<p>In the main directory create a new file called <em><strong>app.py</strong></em> and paste the code below:</p>



<pre class="wp-block-code python"><code>import json
from flask import Flask, request
from syndicai import PythonPredictor

app = Flask(__name__)


@app.route('/')
def hello():
    """ Main page of the app. """
    return "Hello World!"


@app.route('/predict')
def predict():
    """ Return JSON serializable output from the model """
    payload = request.args
    classifier = PythonPredictor("")
    return classifier.predict(payload)


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)</code></pre>



<p>At the begining of the file we import important <code>json</code>, <code>flask</code> as well as a <code>PythonPredictor</code> class form the <em>syndicai.py</em> file.</p>



<p>Next, we create an instance of this class. The first argument is the name of the application’s module or package. This is needed so that Flask knows where to look for templates, static files, and so on. For more information have a look at the <a aria-label="undefined (opens in a new tab)" href="https://flask.palletsprojects.com/en/1.1.x/api/#flask.Flask" target="_blank" rel="noreferrer noopener nofollow">Flask documentation</a>.</p>



<p>We then use the <code>route()</code> decorator to tell Flask what URL should trigger our function.</p>



<p>The function is given a name which is also used to generate URLs for that particular function, and returns the message we want to display in the user’s browser.</p>



<p>Finally, start the web service using <code>python app.py</code> command and paste <code>http://localhost:5000</code> in the web browser to see <em>&#8220;Hello World!&#8221;</em>.</p>



<p>In order to run a model paste <code>http://localhost:5000/predict?url=https://i.imgur.com/PzXprwl.jpg</code> in the browser or run the following REST API request in your terminal. Keep in mind the first model run will take some time because the model will download missing files (weights).</p>



<pre class="wp-block-code bash"><code>curl --request GET \
  --url 'http://localhost:5000/predict?url=https://i.imgur.com/PzXprwl.jpg'</code></pre>



<p>Congrats! You have just created a flask app!</p>



<p class="infoblock">💡 <strong>Explore</strong><br>Learn more about Web services for ML. <br><a href="https://realpython.com/api-integration-in-python/#appendix-rest-in-a-nutshell" target="_blank" rel="noreferrer noopener">REST in a nutshell</a><br><a href="https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world" target="_blank" rel="noreferrer noopener">Flask Mega Tutorial</a> by Miguel Grinberg <br><a href="https://github.com/humiaozuzu/awesome-flask" target="_blank" rel="noreferrer noopener">A curated list of awesome Flask resources and plugins</a><br><a href="https://exploreflask.com/en/latest/organizing.html" target="_blank" rel="noreferrer noopener">Organizing flask project</a></p>



<h2>Wrap with a Docker</h2>



<p>The next step is wrapping a flask service with all dependencies and making it reproducible. </p>



<p>For this purpose, we will use <strong>Docker</strong>. It is an open-source application to create, manage, deploy, and replicate applications using containers. <strong>Containers</strong> can be thought of as a package that houses dependencies required by the app to run at an operating system level. This means that each application deployed using Docker lives in an environment of its own and its requirements are handled separately.</p>



<p>Deploying a Flask app with Docker allows us to replicate the application across different servers with no reconfiguration.</p>



<p>In order to use docker, you need to first create a <em><strong>Dockerfile</strong></em> in the main directory with the following code.</p>



<pre class="wp-block-code docker"><code># Use python as base image
FROM python:3.6-stretch

# Use working directory /app/model
WORKDIR /app/model

# Copy and install required packages
COPY requirements.txt .
RUN pip install --trusted-host pypi.python.org -r requirements.txt

# Copy all the content of current directory to working directory
COPY . .

# Set env variables for Cloud Run
ENV PORT 5000
ENV HOST 0.0.0.0

# Open port 5000
EXPOSE 5000:5000

# Run flask app
CMD &#91;"python","app.py"]</code></pre>



<p><strong>The Dockerfile</strong> is a text document that contains commands used to assemble the image. Paste the code below inside. Creating a Dockerfile it&#8217;s worth to have in mind a couple of best practices:</p>



<ol><li>Layer with <em>requirements.txt</em> installation needs to be above the layer where you copy model files. Thank to that approach, each time when you will change something in your model, docker will rebuild the image without installing dependencies again.</li><li>Use more specific tags and dependency versions. It will prevent you from failure when new library updates are released.</li><li>Change the working directory for your files, don&#8217;t work in the root.</li></ol>



<p>For more detailed information you can read the <a href="https://docs.docker.com/engine/reference/builder/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Dockerfile reference</a>.</p>



<p>Additionally to <em>Dockerfile</em> let&#8217;s create a <em>docker-compose.yaml</em> that will help us test the app locally.</p>



<pre class="wp-block-code yaml"><code>version: '3'
services:

  flask_classifier:
    build: .
    ports:
      - 5000:5000</code></pre>



<p>The docker-compose is a tool for defining and running multi-container Docker applications using <em>yaml</em> file. Compose works in all environments: production, staging, development, testing, as well as CI workflows and it allows very quickly run a dockerized app.</p>



<p>In order to check if the app is working properly inside the docker just run <code>docker-compose up --build</code> in the terminal. The service should be available under the port 5000.</p>



<p class="infoblock">💡 <strong>Explore</strong><br>Please check out the following resources to learn more about Docker.<br><a href="https://docs.docker.com/engine/reference/builder/" target="_blank" rel="noreferrer noopener">Dockerfile reference</a><br><a href="https://phoenixnap.com/kb/docker-image-vs-container" target="_blank" rel="noreferrer noopener">Docker Image vs Container</a><br><a href="https://www.docker.com/blog/intro-guide-to-dockerfile-best-practices/" target="_blank" rel="noreferrer noopener">Intro guide to Dockerfile best practices</a></p>



<h2>Deploy on Cloud Run</h2>



<p>We already exposed our app in the form of a web service and created a Dockerfile that allows us to customize the runtime of our container. At this stage, the repository structure should look as follows.</p>



<pre class="wp-block-code bash"><code>.
├── app.py
├── docker-compose.yaml
├── Dockerfile
├── README.md
├── requirements.txt
├── sample_data
│   ├── input.jpg
│   └── sample_input.json
└── syndicai.py</code></pre>



<p>The last step of this tutorial is service deployment so that it will be out to the world. We will use a <strong>Cloud Run</strong>, a serverless platform for hosting docker containers. It is fully manageable so you don&#8217;t need to worry about the infrastructure.</p>



<p>In order to deploy a service, you need to run two commands. Remember to substitute PROJECT_ID with the name of your GCP project!</p>



<pre class="wp-block-code bash"><code># Build a docker image 
gcloud builds submit --tag gcr.io/PROJECT_ID/flask-classifier

# deploy a docker image to cloud run
gcloud run deploy --image gcr.io/PROJECT_ID/flask-classifier \
  --platform managed \
  --port 5000 \
  --memory 1G</code></pre>



<p>You can see the deployed service in the Cloud Run.</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-1024x523.png" alt="" class="wp-image-510" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-1024x523.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-300x153.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1-768x393.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/cloudrun1.png 1485w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure></div>



<p>In order to run your model, just copy the url of the service and use the following request.</p>



<pre class="wp-block-code"><code>curl --request GET \
  --url 'https://CLOUD_RUN_SERVICE_URL/predict?url=https://i.imgur.com/PzXprwl.jpg'</code></pre>



<p>Wow! You have just deployed the Image Classifier!</p>



<div class="wp-block-image justify-center"><figure class="aligncenter size-large"><img src="https://media.giphy.com/media/xDpB3lRInUYla/giphy.gif" alt="flask mlops syndicai"/></figure></div>



<h2>Conclusion</h2>



<p>In the following tutorial, you had a chance to explore one approach of AI model deployment, based on flask, docker, and Cloud Run.</p>



<p>The solutions that we&#8217;ve tried has couple pros and cons.</p>



<p>On one hand, flask gives you a lot of freedom, is easy to use, has integrated unit testing support and extensive documentation. However, a lot of freedom means that you need to configure a lot of things to make it work really well in production.</p>



<p class="text-center">* * *</p>



<p>If you found that tutorial helpful, or have some questions, please let us know via <a href="mailto:hello@syndicai.co" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">mail</a> or join us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">slack</a>. We would love to hear your feedback.</p>



<p>We would also like to inspire you via our <a aria-label="undefined (opens in a new tab)" href="https://showcase.syndicai.co/" target="_blank" rel="noreferrer noopener">AI models Showcase</a> page and give you a warm invitation to try out our <a href="https://syndicai.co/">MLOps platform</a> that speeds up the work of AI Teams 10x.</p>


Deploy ML model using Flask, Docker, and Cloud Run


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/ai_project_canvas.png" alt="ai project canvas by syndicai" class="wp-image-493" title="ai project canvas syndicai" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/ai_project_canvas.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/01/ai_project_canvas-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/ai_project_canvas-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">AI Project Canvas will help you succeed with the next AI initiative. Visualize important components, activities, and potential risks long before the first line of code. </p>



<h2>Why I need this?</h2>



<p>As a deep-tech consultant, I have noticed that Project Managers struggle with IT projects powered by AI. That causes a very high rate of failed projects that don&#8217;t go beyond the piloting stage. One of the main reasons why they have problems is a lack of awareness that AI projects are very different from traditional IT projects.</p>



<p>I found that there is a need for a tool that might help managers collect and combine all major components of the whole project in one place. Looking at the bigger picture might help AI teams explore the main goal, potential scope, and difficulties long before writing the first line of code.</p>



<p>Taking inspiration from a Business Model Canvas, created by Alexander Osterwalder I have build an AI Project Canvas.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-1024x723.png" alt="ai project canvas" class="wp-image-494" title="ai project canvas" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-1024x723.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-300x212.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-768x543.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-1536x1085.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-2048x1447.png 2048w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_1-1568x1108.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Project Canvas</figcaption></figure>



<h2>What is an AI Project canvas?</h2>



<p>An <strong><a href="https://docs.google.com/document/d/1o7V8qtVML1N1n3ghCKdgHg5iBm0ZjCrLnjAYbc-wQg8/edit?usp=sharing" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">AI Project Canvas</a></strong> is a detailed plan for Team Leaders, Project Managers who create products and services powered by AI.</p>



<p><strong>It consists of</strong> nine building blocks that combined together create a three-stage workflow <em>Build-Deploy-Manage</em>. The <em>Build</em> part relates to the Exploratory Data Analysis that needs to be performed in order to extract information from data. The <em>Deploy</em> part is to put the Machine Learning model into production. The <em>Manage</em> part is related to human and financial resources need to finish the project with success.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-1024x723.png" alt="ai project canvas components" class="wp-image-495" title="ai project canvas components" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-1024x723.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-300x212.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-768x543.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-1536x1085.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-2048x1447.png 2048w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_2-1568x1108.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Project Canvas consists of three major parts: Build, Deploy, and Manage</figcaption></figure>



<p><strong>It is worth to use it</strong> each time when you &#8230;</p>



<ol><li>Create a concept and plan a roadmap of the project</li><li>Gather proper resources and estimate cost structure</li><li>Need to convenience the management to fund the idea</li></ol>



<p>Inspired? Let&#8217;s explore the whole concept in detail!</p>



<h2>How to use it?</h2>



<p>Spent <strong>no more than 30min</strong> to fill a canvas. The main goal is to give you a basic understanding of the project and familiarize you with all the missing puzzles. Is not a detailed plan!</p>



<p>Follow the sample questions below to get the best experience.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-1024x723.png" alt="ai project canvas" class="wp-image-496" title="ai project canvas" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-1024x723.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-300x212.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-768x543.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-1536x1085.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-2048x1447.png 2048w, https://data.syndicai.co/wp-content/uploads/2021/01/canvas_3-1568x1108.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Project Canvas steps</figcaption></figure>



<h3>1. Concept</h3>



<ol><li>What is the problem you are trying to solve? </li><li>What is the solution you want to build? </li><li>How big is the impact on the business of that project? Who is it for, and will benefit most?</li></ol>



<h3>2. Data</h3>



<ol><li>What internal and external Data Sources do you need?</li><li>What databases, tables you want to use?</li><li>How will you connect to Data Sources?</li></ol>



<h3>3. Model</h3>



<ol><li>What is the type of the task (prediction, classification, &#8230;)?</li><li>Which model architecture you want to use?</li><li>What will be the input/output of the model?</li></ol>



<h3>4. Metrics</h3>



<ol><li>How will you evaluate data correctness?</li><li>What metrics will you use for models, and how will you evaluate them? </li><li>What is your business KPI of success (Minimum Justifiable Improvement)? </li><li>How will you track progress?</li></ol>



<h3>5. Infrastructure</h3>



<ol><li>What tools and services do you want to use to perform operations on data, models, and code? </li><li>Where will you deploy the whole system &#8211; on cloud or on-prem?</li></ol>



<h3>6. Pipeline</h3>



<ol><li>How will you connect all services, and what will be the workflow? </li><li>Which parts and how will be automated? </li><li>Will you run a model in the batch or real-time?</li></ol>



<h3>7. Maintenance</h3>



<ol><li>How will you take care of Versioning, Reproducibility, Testing, and Monitoring of Data, Model, and Code?</li></ol>



<h3>8. Team</h3>



<ol><li>Who will be the end-user of the product? </li><li>Who will benefit from the product?</li><li>How customer will benefit from the outcome?</li><li>Who is needed to perform the Build &amp; Deploy phase?</li></ol>



<h3>9. Costs</h3>



<ol><li>What we have to pay for?</li><li>How much do we have to pay?</li><li>What is the expected revenue, or non-financial added value?</li><li>What is the problem you are trying to solve? </li></ol>



<h2>Summing up</h2>



<p>AI is a great technology that empowers more and more companies. However, it important to be aware that AI-powered projects differ from traditional software engineering projects.</p>



<p>Introduced <strong>AI Project Canvas aims to help grasp a big picture of the project in a fast and easy way.</strong> We highly encourage you to use a template before each AI project so that you will have a general overview of all potential difficulties and weaknesses that might occur during sprints.</p>



<p>If you found that article inspiring or have some questions, please let us know via <a href="mailto:hello@syndicai.co" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">mail</a> or join us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">slack</a>.</p>



<p class="has-text-align-center text-center">* * *  </p>



<p>We would also like to inspire you via our <a href="https://showcase.syndicai.co/">AI model Showcase</a> page and give you a warm invitation to try out Syndicai <a href="/">MLOps</a> Platform for AI deployment &amp; integration that speeds up the work of AI Teams 10x.</p>


AI Project Canvas: The best way to deploy AI in the organization


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/clip.png" alt="clip by openai" class="wp-image-483" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/clip.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/01/clip-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/clip-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<h2>What is CLIP?</h2>



<p>Lately OpenAI introduced a new <a href="https://www.youtube.com/channel/UCobqgqE4i5Kf7wrxRxhToQA">multimodal</a> neural network called CLIP (Contrastive Language-Image Pre-training). A model trained on 400 million pairs of images and text collected from the internet. It uses zero-shot learning capabilities akin to GPT-2 and GPT-3 language models. It is able to perform image classification based on natural language.</p>



<p><em>“We find that CLIP, similar to the GPT family, learns to perform a wide set of tasks during pretraining, including object character recognition (OCR), geo-localization, action recognition, and many others.”</em>  &#8211; a quote from OpenAI <a aria-label="undefined (opens in a new tab)" href="https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language.pdf" target="_blank" rel="noreferrer noopener nofollow">paper</a>.</p>



<p>CLIP is very efficient in a variety of image classification tasks. However, it struggles with domain tasks such as pulmonary nodules classification or plant pathogens detections.</p>



<h2>Why is it so great?</h2>



<p>Currently, the computer vision space needs to tackle many challenges with datasets labeling (that is costly and labor-intense), and problems with model adaptations to different tasks. </p>



<p>CLIP by OpenAI aims to solve those challenges.</p>



<p>The new model architecture allows to <strong>train a model using detailed descriptions</strong>, not a single words such as labels. In this particular case, the authors of the model took descriptions from internet image captions. </p>



<p>Thanks to the similar “zero-shot” capabilities of GPT-2 and GPT-3 CLIP <strong>can be applied to any visual classification benchmark</strong>. The only thing you need to do is to provide names of the visual categories. </p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/clip_1.png" alt="clip versus traditional image classification models" class="wp-image-484" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/clip_1.png 893w, https://data.syndicai.co/wp-content/uploads/2021/01/clip_1-298x300.png 298w, https://data.syndicai.co/wp-content/uploads/2021/01/clip_1-150x150.png 150w, https://data.syndicai.co/wp-content/uploads/2021/01/clip_1-768x772.png 768w" sizes="(max-width: 893px) 100vw, 893px" /><figcaption>CLIP vs ImageNet</figcaption></figure>



<h2>Explore yourself</h2>



<p>In order to run a model, go to the <a aria-label="undefined (opens in a new tab)" href="https://github.com/mlaskowski17/clip" target="_blank" rel="noreferrer noopener">repository</a>. Explore how a zero-shot prediction using CLIP works in practice running a notebook.</p>



<p>For the purpose of the quick tour, we tried to classify the image of the shark, but only partially seen. We used classes from the Cifar100.</p>



<div class="wp-block-image"><figure class="aligncenter size-medium"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/shark-200x300.jpg" alt="clip shark" class="wp-image-485" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/shark-200x300.jpg 200w, https://data.syndicai.co/wp-content/uploads/2021/01/shark-683x1024.jpg 683w, https://data.syndicai.co/wp-content/uploads/2021/01/shark-768x1152.jpg 768w, https://data.syndicai.co/wp-content/uploads/2021/01/shark-1024x1536.jpg 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/shark.jpg 1280w" sizes="(max-width: 200px) 100vw, 200px" /><figcaption>Input image that was used to run a model</figcaption></figure></div>



<p>As a result we got the following output.</p>



<pre class="wp-block-code"><code>Top predictions:

         dolphin: 57.62%
           shark: 36.93%
           whale: 2.86%
   aquarium_fish: 0.83%
             sea: 0.46%</code></pre>



<p>We can see that results are quite good. Shark is in second place. You can further explore a model or deploy it on <a href="/">Syndicai</a>. For more computer vision samples go to <a href="https://showcase.syndicai.co">showcase</a> page.</p>


What exactly is CLIP by OpenAI and why is it so great?


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/DDI_mask.png" alt="detect face mask " class="wp-image-474" title="Syndicai face mask detector" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/DDI_mask.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/01/DDI_mask-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/DDI_mask-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">More and more companies struggle with a difficult Covid situation. In the following tutorial, we would like to show you a simple, and fast way to build a face mask detector app.</p>



<p></p>



<h2>Introduction</h2>



<p>In the following tutorial, we will show you how to <strong>Develop, Deploy, </strong>and<strong> Integrate</strong> a Keras <a href="https://github.com/syndicai/models/tree/master/keras/face_mask_detector" target="_blank" rel="noreferrer noopener">Face Mask Detector</a> algorithm using <a href="/">Syndicai </a>Platform. For this purpose, we will use <a aria-label="undefined (opens in a new tab)" href="https://github.com/chandrikadeb7/Face-Mask-Detection/blob/master/LICENSE" rel="noreferrer noopener nofollow" target="_blank">Chandrika Deb</a> implementation that is one of the best available on Github. The main goal of that algorithm is to detect whether a person in the picture is wearing a mask, or not. If you would like to try the state of the art of the basic object detection algorithm, check out how to <a href="/blog/deploy-yolov5/">deploy yolov5</a>.</p>



<p>Let&#8217;s start!</p>



<h2>Step 1 | Develop<a href="#step-1-or-develop"></a></h2>



<p>In the first phase of the tutorial, we need to prepare a model for deployment. In order to do it in the right way, the best way is to follow the checklist below.</p>



<h3>1. Train your model</h3>



<p>Before you start model preparation you need to be sure that your model is trained. In our case, a model is already trained so we don&#8217;t have to go through that task. However, if you would like to explore object detection training workflow, please go to <a href="https://data.syndicai.co/blog/train-yolov5/">train yolov5</a> algorithm.</p>



<h3>2. Create a Git repository</h3>



<p>Your main model files need to be placed in a git repository. However, if you have large files (e.g. with model weights) you can store them in a separate storage or drive so that <a href="https://syndicai.co">Syndicai</a> will download them during model initialization.</p>



<p>In our model storing in separate storage is not necessary, because MobileNetV2 architecture is very lightweight and can be easily placed in a <a href="https://github.com/syndicai/models/tree/master/keras/face_mask_detector/model" target="_blank" rel="noreferrer noopener">Github repository directly</a>.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo-1024x517.png" alt="repository with a face mask detector" class="wp-image-475" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo-1024x517.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo-768x387.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo-1536x775.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo-1568x791.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_repo.png 1895w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Repository with a Face Mask Detector</figcaption></figure>



<h3>3. Add syndicai.py &amp; requirements.txt</h3>



<p>In order to deploy a model with Syndicai you need to have <code>syndicai.py</code> and <code>requirements.txt</code> placed in your main directory.</p>



<p><code>syndicai.py</code> is the main file with the <code>PythonPredictor</code> python class responsible for model prediction. It <a href="/workflow/develop#model-preparation">consist of two methods</a> <code>__init__</code>and <code>predict</code> that are necessary to properly run a model.</p>



<pre class="wp-block-code python"><code>import os
import cv2
import numpy as np

from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.models import load_model

from utils import url_to_image, b64_to_image, image_to_base64


args = {
    "image": "sample_data/out.jpg",
    "face": "face_detector",
    "model": "model/mask_detector.model",
    "confidence": 0.5,
}


class PythonPredictor:

    def __init__(self, config):
        # load our serialized face detector model from disk
        print("&#91;INFO] loading face detector model...")
        prototxtPath = os.path.sep.join(&#91;args&#91;"face"], "deploy.prototxt"])
        weightsPath = os.path.sep.join(&#91;args&#91;"face"],
                                        "res10_300x300_ssd_iter_140000.caffemodel"])
        self.net = cv2.dnn.readNet(prototxtPath, weightsPath)

        # load the face mask detector model from disk
        print("&#91;INFO] loading face mask detector model...")
        self.model = load_model(args&#91;"model"])

    def predict(self, payload):

        # Get image from url
        try: 
            image = url_to_image(payload&#91;"image_url"])
        except:
            image = b64_to_image(payload&#91;"image_b64"])
        orig = image.copy()
        (h, w) = image.shape&#91;:2]

        # construct a blob from the image
        blob = cv2.dnn.blobFromImage(image, 1.0, (300, 300),
                                     (104.0, 177.0, 123.0))

        # pass the blob through the network and obtain the face detections
        print("&#91;INFO] computing face detections...")
        self.net.setInput(blob)
        detections = self.net.forward()

        # loop over the detections
        for i in range(0, detections.shape&#91;2]):
            # extract the confidence (i.e., probability) associated with
            # the detection
            confidence = detections&#91;0, 0, i, 2]

            # filter out weak detections by ensuring the confidence is
            # greater than the minimum confidence
            if confidence > args&#91;"confidence"]:
                # compute the (x, y)-coordinates of the bounding box for
                # the object
                box = detections&#91;0, 0, i, 3:7] * np.array(&#91;w, h, w, h])
                (startX, startY, endX, endY) = box.astype("int")

                # ensure the bounding boxes fall within the dimensions of
                # the frame
                (startX, startY) = (max(0, startX), max(0, startY))
                (endX, endY) = (min(w - 1, endX), min(h - 1, endY))

                # extract the face ROI, convert it from BGR to RGB channel
                # ordering, resize it to 224x224, and preprocess it
                face = image&#91;startY:endY, startX:endX]
                face = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
                face = cv2.resize(face, (224, 224))
                face = img_to_array(face)
                face = preprocess_input(face)
                face = np.expand_dims(face, axis=0)

                # pass the face through the model to determine if the face
                # has a mask or not
                (mask, withoutMask) = self.model.predict(face)&#91;0]

                # determine the class label and color we'll use to draw
                # the bounding box and text
                label = "Mask" if mask > withoutMask else "No Mask"
                color = (0, 255, 0) if label == "Mask" else (0, 0, 255)

                # include the probability in the label
                label = "{}: {:.2f}%".format(
                    label, max(mask, withoutMask) * 100)

                # display the label and bounding box rectangle on the output
                # frame
                cv2.putText(image, label, (startX, startY - 10),
                                    cv2.FONT_HERSHEY_SIMPLEX, 0.45, color, 2)
                cv2.rectangle(
                    image, (startX, startY), (endX, endY), color, 2)

        return image_to_base64(image)d</code></pre>



<p><code>requirements.txt</code> – a file with all libraries and frameworks needed to recreate the model’s environment</p>



<pre class="wp-block-code text"><code>tensorflow==1.15.2
keras==2.3.1
imutils==0.5.3
numpy==1.18.2
opencv-python==4.2.0.*
matplotlib==3.2.1
argparse==1.1
scipy==1.4.1
scikit-learn==0.23.1
pillow==7.2.0</code></pre>



<p>For the Face Mask Detector, we also need other functions and files, so you can explore them in the <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/models/tree/master/keras/face_mask_detector" target="_blank" rel="noreferrer noopener">main repository</a>.</p>



<p>It is important to notice that model weights are initialized in the <code>init</code> method which is run only once, before the model is in the <a href="/workflow/manage/model-statuses">Active</a> state.</p>



<p></p>



<h2>Step 2 | Deploy</h2>



<p>In order to deploy a model, you need connect your repo to the Syndicai Platform, so that the whole infrastructure will be created.</p>



<h3>1. Connect a git repository</h3>



<p>Go to <a href="https://app.syndicai.co/" target="_blank" rel="noreferrer noopener">Syndicai Platform</a> and press New Model on the &#8220;Overview&#8221; page. You will be redirected to a short form. Place the URL for your model (In our case it is: <a href="https://github.com/syndicai/models/tree/master/keras/face_mask_detector" target="_blank" rel="noreferrer noopener">https://github.com/syndicai/models/tree/master/keras/face_mask_detector</a>)</p>



<p>Press &#8220;Continue&#8221; and &#8220;Deploy&#8221; in the next card.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/keras_new-1024x515.png" alt="Deploy a model via Syndicai Platform" class="wp-image-476" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/keras_new-1024x515.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_new-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_new-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_new-1536x772.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_new-1568x789.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_new.png 1895w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>When you press &#8216;New Model&#8217; you will be redirected to a quick form to Connect a repo.</figcaption></figure>



<h3>2. Wait for the Active model</h3>



<p>Now the whole magic happens. Syndicai creates a webservice, builds a docker, and run a model on the scalable Kubernetes cluster automatically so that you don&#8217;t need to care about the infrastructure setup. You can go to the Model &#8220;Builds&#8221; section to see logs from the build.</p>



<p>You need to have a build with status <a href="/workflow/manage/build-statuses">success</a> and model with status <a href="/workflow/manage/model-statuses">Active</a>.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/keras_active-1024x514.png" alt="Face Mask Detection algorithm deployed via Syndicai" class="wp-image-477" title="Deployed Face Mask Detector model" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/keras_active-1024x514.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_active-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_active-768x385.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_active-1536x771.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_active-1568x787.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_active.png 1895w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>When build will turn into success state you will notice that your model is Active.</figcaption></figure>



<p></p>



<h2>Step 3 | Integrate</h2>



<p>Congratulations, You have deployed your model!</p>



<p>Now you can go to the model page and Run it on the platform in the &#8220;Run a model&#8221; section. Our <code>syndicai.py</code> file was written in a way that can handle two different image input formats: <a href="https://github.com/syndicai/models/blob/master/keras/face_mask_detector/sample_data/sample_input_url.json" target="_blank" rel="noreferrer noopener">URL</a> or <a href="https://github.com/syndicai/models/blob/master/keras/face_mask_detector/sample_data/sample_input_b64.json" target="_blank" rel="noreferrer noopener">base64</a>. </p>



<pre class="wp-block-code json"><code>{
	"image_url": "https://bsmedia.business-standard.com/_media/bs/img/article/2020-07/12/full/1594519569-3012.JPG"
}</code></pre>



<pre class="wp-block-code json"><code>{
	"image_b64": "/9j/4AAQSkZJRgABAQAAAQABA..."
}</code></pre>



<p>You can copy one of them and paste it in the &#8220;Run a model&#8221; section.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/keras_run-1024x515.png" alt="Run a Face Mask detection model" class="wp-image-478" title="Face Detection Algorithm deployed" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/keras_run-1024x515.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_run-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_run-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_run-1536x772.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_run-1568x789.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/keras_run.png 1893w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Paste a sample input JSON in order to run a model</figcaption></figure>



<p>When a model is working properly you can go to the &#8220;Integrate&#8221; subpage and use REST API to integrate a model with a frontend page. For the frontend page, you can use React, Nextjs or other frameworks. However, you can also try our ready-made template with <a href="https://showcase.syndicai.co/mask-detector">a Face Mask Detector showcase</a>. The whole code for that <a href="https://github.com/syndicai/showcase">demo page is open-source</a> so you can fork and play with it as you like.</p>



<p></p>



<h2>Closing remarks</h2>



<p>The main goal of that tutorial was first of all to inspire you and then show that you can deploy  any trained AI model in minutes. No config needed, no deep tech knowledge required &#8211; just a simple way from a model to actual use case with a click of a button. </p>



<p>If you like this kind of content or would like to see how to deploy some particular mode please, <a href="mailto:hello@syndicai.co">drop us a line</a> or find us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw">slack</a>!</p>


How to build face mask detector application?


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/ecommerce_2021.png" alt="Top 3 AI models for e-commerce in 2021" class="wp-image-464" title="AI in ecommerce in 2021" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/ecommerce_2021.png 1000w, https://data.syndicai.co/wp-content/uploads/2021/01/ecommerce_2021-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/ecommerce_2021-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Explore the top 3 AI algorithms for eCommerce that are going o be a game-changer in 2021. Visual search, Content generation, and Hyper recommendation.</p>



<h2>Visual Search</h2>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-1024x520.jpg" alt="fashion visual search" class="wp-image-465" title="visual search is getting traction in the space of eCommerce" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-1024x520.jpg 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-300x152.jpg 300w, https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-768x390.jpg 768w, https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-1536x780.jpg 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-2048x1040.jpg 2048w, https://data.syndicai.co/wp-content/uploads/2021/01/visual_search_1-1568x796.jpg 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Fashion Visual Search introduced by IBM in 2020, source: https://cognitivefashion.github.io/portfolio/api_visual_search/</figcaption></figure>



<p>You go to the shopping center and see an astonishing leather jacket. Unfortunately, I had no time to try it on so I only took a picture through a in the shop window. After I come home I spent a couple of hours looking for that jacket with no success.</p>



<p>During daily life, we encounter many situations during which we are looking for a particular product. Whether that&#8217;s a new car, an astonishing dress, or the vegetable I want to weigh in the supermarket to get a sticker.</p>



<p>Can you imagine how great would it be to get what you want based on some picture? You don&#8217;t have to do it because AI is capable of all those things. The whole concept is called Visual Search and it is going to be a game-changer in the whole eCommerce industry. </p>



<p>Visual search uses real-world images such as photographs, internet images, or screenshots as an input for online searches. The main goal of those systems is to understand the content with the context of the image and return a list of products. The most common applications are in the space of home decor retailers and fashion. It helps to suggest products that are hard to describe in written language.</p>



<p></p>



<h2>Content generation</h2>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3-1024x516.png" alt="GPT-2 content generation" class="wp-image-466" title="GPT-3 is on the of most advanced AI algorithms in the sapce of NLP" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3-1024x516.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3-768x387.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3-1536x774.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3-1568x790.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/gpt-3.png 1903w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Sample article about AI content generation, source: https://vasilishynkarenka.com/gpt-3/</figcaption></figure>



<p>One of the most popular concepts among marketers is SEO optimization. The main goal is to get traction on your website, social media, and other platforms. You can do it in many ways, the most popular one is pure content generation (articles, blog posts, product descriptions, tweets, posts, etc.). The more content you generate, the higher will be the chance that your potential customer will find it.</p>



<p>However, for many companies content creation is a time-consuming process that is why the solution might be Content Generation algorithms.</p>



<p>They can produce tweets, posts, or even a whole article for you in just seconds! The whole <a href="https://data.syndicai.co/blog/machine-learning-operations/">machine learning operations</a> workflow is highly optimized and is capable to generate content targeted directly to a specific customer in real-time.</p>



<p>The perfect example of such an algorithm is <a href="https://www.youtube.com/watch?v=_x9AwxfjxvE" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">GPT-3</a>. The algorithm introduced by OpenAI able to answer questions, reading comprehension, or even summarize text.</p>



<p></p>



<h2>Hyper personalization</h2>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/header-8-1024x512.png" alt="netflix personalization system" class="wp-image-467" title="Netflix recommendation system" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/header-8-1024x512.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/header-8-300x150.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/header-8-768x384.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/header-8.png 1200w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Netflix has one the most advanced AI systems for eCommerce, source: https://slideslive.com/38917692/recent-trends-in-personalization-a-netflix-perspective</figcaption></figure>



<p>It is widely known that more and more eCommerce companies are empowered with 1-to-1 personalization. It is a practice of delivering a unique experience in the for of product recommendation or omnichannel personalization based on shopping data (previously seen products, past orders). In order to provide such a service, the whole system needs to work in real-time experience which done by highly optimized Machine Learning workflows. </p>



<p>However, today&#8217;s marketing strategies extend far beyond traditional approaches. In order to increase the conversion rate of every single potential customer a new concept was introduced &#8211; Hyper-personalization.</p>



<p>The main goal of Hyper-personalisation is to collect not only shopping data but also 3rd parties information about the customer such as past locations, buying preferences, habits, behavioral data, etc. Everything in order to generate the content that perfectly matches your shopping needs. For example catchy title and description of the product that correlates to your hobby and passion.</p>



<p></p>



<p>Check out our <a href="https://showcase.syndicai.co/">Showcase</a> page with some AI examples and <a href="mailto:hello@syndicai.co">drop us a line</a> if you want more of this type of content.</p>


Top 3 AI algorithms for eCommerce in 2021


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_tensorflow.png" alt="deploy tensorflow in minutes" class="wp-image-436" title="deploy tensroflow at scale" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_tensorflow.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_tensorflow-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_tensorflow-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<h2>Introduction</h2>



<p>As we all know AI is the major driving force behind many of the applications we see today. However, in order to put that AI to those applications, we need to go through an <a href="https://www.ibm.com/cloud/learn/exploratory-data-analysis" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Exploratory Data Analysis</a> (EDA) phase, and when a model is working deploy &amp; integrate it with the app. When it comes to the last step (deployment &amp; integration) it is not an easy task if you want to make it ready for production.</p>



<p>What if there would be an easy and fast way to go from a trained model to actual use-case with a click of a button? What if that process would be a production-ready one?</p>



<p>In this article, we will show you how to deploy a TensorFlow model in three quick steps.</p>



<h2>Step 1: Develop a Tensorflow model</h2>



<p>For the purpose of this tutorial, we will deploy a <a aria-label="undefined (opens in a new tab)" href="https://arxiv.org/abs/1705.06830" target="_blank" rel="noreferrer noopener nofollow">Golnaz and Honglak</a> implementation of the <a href="https://github.com/syndicai/models/tree/master/tensorflow/style_transfer">Style Transfer</a> (also called Neural Style Transfer).</p>



<p>Neural style transfer is an optimization technique that takes as input two images (a content image with style reference one) and combines them together so the output image looks like it&#8217;s “painted” in the style of the reference image.</p>



<p>The newest approach presented in the paper investigates a method that combines the flexibility of the neural algorithm of artistic style with the speed of fast style transfer networks so that the algorithm is much faster, and can work in real-time.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/style_transfer.png" alt="deploy style transfer" class="wp-image-441" title="deploy neural style transfer" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/style_transfer.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/style_transfer-300x118.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/style_transfer-768x302.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /><figcaption>The input of the model is Content image and Style reference, while the output is the combination both images.</figcaption></figure>



<p>We have already prepared the <a href="https://github.com/syndicai/models/tree/master/tensorflow/style_transfer">Style Transfer</a> model on our GitHub repository. You don’t have to do anything with the repository for now.</p>



<h2>Step 2: Deploy a Tensorflow model</h2>



<p>In the traditional workflow of Machine Learning model deployment, you need to go through several steps (create a webservice, build a docker and serve on Kubernetes cluster).</p>



<p><a href="/">Syndicai</a> takes care of all those steps. You just need to connect the git repository with your model and the REST API will be created automatically with one click. Moreover, Syndicai takes care of the scalability of resources. The resulting API offers great flexibility because you can connect it to any device.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1.png" alt="deploy ai at scale" class="wp-image-377" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1-300x78.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1-768x200.png 768w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI model deployment in the traditional way (on top) vs Syndicai way (on the bottom)</figcaption></figure>



<p>You can also try out to <a href="https://data.syndicai.co/blog/deploy-keras-model/">deploy a Keras model</a>.</p>



<h3>Prepare a repository</h3>



<p>When model Apart from putting your model in the GitHub repository, you have to upload two additional files there: <code>requirements.txt</code> and <code>syndicai.py</code>.</p>



<p><code>requirements.txt</code> – a file with all libraries and frameworks needed to recreate model’s environment</p>



<pre class="wp-block-code text"><code>numpy==1.19.4
matplotlib==3.2.2
Pillow==7.0.0
tensorflow==2.4.0
tensorflow-hub==0.10.0</code></pre>



<p><code>syndicai.py</code> – main file with the <code>PythonPredictor</code> class responsible for model prediction. </p>



<pre class="wp-block-code python"><code>import os
import io
import base64
import functools

from PIL import Image
import numpy as np
import tensorflow as tf
import tensorflow_hub as hub

from helpers import *


class PythonPredictor:

    def __init__(self, config):
        # Define style image
        self.style_image_url = 'https://upload.wikimedia.org/wikipedia/commons/c/c5/Edvard_Munch%2C_1893%2C_The_Scream%2C_oil%2C_tempera_and_pastel_on_cardboard%2C_91_x_73_cm%2C_National_Gallery_of_Norway.jpg'
        # Import TF-Hub module
        hub_handle = 'https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2'
        self.hub_module = hub.load(hub_handle)

    def predict(self, payload):
        # Define content image
        content_image_url = payload&#91;"url"]

        # Load images
        content_img_size = (500, 500)
        style_img_size = (300, 300)

        style_image = load_image(self.style_image_url, style_img_size)
        content_image = load_image(content_image_url, content_img_size)
        style_image = tf.nn.avg_pool(
            style_image, ksize=&#91;3, 3], strides=&#91;1, 1], padding='SAME')

        # Stylize content image with given style image.
        outputs = self.hub_module(tf.constant(content_image),
                                  tf.constant(style_image))
        stylized_image = outputs&#91;0]

        # get PIL image and convert to base64
        img = Image.fromarray(np.uint8(stylized_image.numpy()&#91;0] * 255))
        im_file = io.BytesIO()
        img.save(im_file, format="PNG")
        im_bytes = base64.b64encode(im_file.getvalue()).decode("utf-8")

        return im_bytes</code></pre>



<p>These two files are necessary for the Syndicai to be able to recreate the environment and know which function to use for prediction.</p>



<h3>Connect the repository to Syndicai</h3>



<p>When we have the GitHub repository with <code>requirements.txt</code> and <code>syndicai.py</code> ready, we can proceed to connect it to the Syndicai platform. In order to that, go to <a href="https://app.syndicai.co/">https://app.syndicai.co/</a>, login, click New Model on the Overview page. You will be redirected to the quick form. Follow the steps, and as soon as you finish, the infrastructure will start building. You will need to wait a couple of minutes for the model to become <a href="https://docs.syndic.ai/workflow/manage/model-statuses">Active</a>.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/style_01-1024x514.png" alt="deploy style transfer" class="wp-image-450" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/style_01-1024x514.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/style_01-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/style_01-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/style_01-1536x772.png 1536w, https://data.syndicai.co/wp-content/uploads/2021/01/style_01-1568x788.png 1568w, https://data.syndicai.co/wp-content/uploads/2021/01/style_01.png 1905w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Deployed Style Transfer model should have build status &#8220;success&#8221; and badge &#8220;Active&#8221; next to the name</figcaption></figure>



<p>For more information about the model preparation or deployment process go to <a href="https://docs.syndic.ai/">Syndicai Docs</a>.</p>



<h2>Step 3: Integrate a Tensorflow model</h2>



<p>You&#8217;ve done it!</p>



<p>Your model is deployed, and your REST API is ready. In order to perform a quick test, just copy &amp; paste a sample input script in the model run section.</p>



<pre class="wp-block-code json"><code>{
    "url": "https://upload.wikimedia.org/wikipedia/commons/d/d7/Green_Sea_Turtle_grazing_seagrass.jpg"
}</code></pre>



<p>Remember that model needs to be <a href="https://docs.syndic.ai/workflow/manage/model-statuses">Active</a> to make it work!</p>



<p>If everything works fine, you can now connect the API with any device or service. As an example, you can go to the <a href="https://showcase.syndicai.co/">Showcase page</a> and explore the sample implementation of the model.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style-1024x515.png" alt="" class="wp-image-444" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style-1024x515.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style-300x151.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style-768x386.png 768w, https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style-1536x772.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style-1568x788.png 1568w, https://data.syndicai.co/wp-content/uploads/2020/12/tensorflow_style.png 1894w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Style Transfer implementation: https://showcase.syndicai.co/style-transfer</figcaption></figure>



<h2>Summary</h2>



<p>You had a chance to see how to deploy tensorflow model in minutes. Syndicai allows you to deploy and integrate AI models at scale in a simple and fast way. You don&#8217;t need to setup the infrastructure or take care of scalability, Syndicai Platform will do it for you.</p>



<p>If you found that useful, or you want to get more of those types of tutorials – please drop us a line by mail or catch us on slack.</p>


How to deploy a Tensorflow model in minutes?


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_gpt2.png" alt="" class="wp-image-409" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_gpt2.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_gpt2-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_gpt2-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Let&#8217;s explore a simple and fast way to deploy GPT-2 model at scale in minutes without docker and Kubernetes setup. We also have a tutorial on how to <a href="https://syndicai.co/blog/deploy-pytorch-model">deploy a PyTorch model</a>.</p>



<h2>Introduction</h2>



<p>Do you know what’s better than AI models in PyCharm or Jupyter Notebook? AI models working on production! It feels great when anyone can use your model, but there are many potential <a href="https://syndicai.co/blog/problem-with-ai-model-deploymente">problems with AI model deployment</a>.</p>



<p>We will need to create a web service with Flask, recreate the environment in Docker, set up the infrastructure, and deploy the model to the Google Cloud or Amazon AWS, right? Fortunately not! This is how <a href="https://syndicai.co/blog/machine-learning-operations">Machine Learning Operations</a> looked like. In this tutorial, I will show you how to deploy a GPT-2 model with one tool called <a href="https://syndicai.co/">Syndicai</a> in a few simple clicks.</p>



<h2>Step 1: Develop a GPT-2 model</h2>



<p>Okay, but before we deploy anything, we need some cool AI model. We will use one of the hottest NLP models straight out of the OpenAI labs &#8211; <a href="https://openai.com/blog/better-language-models/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">GPT-2</a>. If you already have your proprietary model ready, then you can skip this section.</p>



<p>GPT-2 is a “transformer-based language model with 1.5 billion parameters trained on a dataset of 8 million webpages”.&nbsp; The main goal of the model is to predict the next word given the collection of previous words. It achieved state-of-the-art results (now suppressed by GPT-3) on a variety of different datasets. The most amazing thing about it is the fact that it wasn’t trained on any domain-specific NLP task. Nevertheless, it was superior to other hand-crafted models. This way of comparing the performance of the model is called “zero-shot”.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/gpt2-1024x645.png" alt="GPT-2 model completion task" class="wp-image-255" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/gpt2-1024x645.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/gpt2-300x189.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/gpt2-768x484.png 768w, https://data.syndicai.co/wp-content/uploads/2020/12/gpt2.png 1049w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>GPT-2 Model Completion Task: <a href="https://openai.com/blog/better-language-models/#sample2" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow"><em>https://openai.com/blog/better-language-models/#sample2</em></a></figcaption></figure>



<p>We have already prepared the GPT-2 model on our <a href="https://github.com/syndicai/models/tree/master/pytorch/gpt2">GitHub repository</a>. You don’t have to do anything with the repository for now.</p>



<h2>Step 2: Deploy GPT-2 model</h2>



<p>Let&#8217;s find out how to skip all the repetitive steps in the deployment process. Enter <a href="/">Syndicai</a> &#8211; the tool that takes a GitHub repository and returns the Rest API. Under the hood, Syndicai setups the entire infrastructure with one click. Moreover, it takes care of the scalability of resources. The resulting API offers great flexibility because you can connect it to any device.  </p>



<p></p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267.png" alt="ai model deployment traditional approach vs syndicai" class="wp-image-247" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-300x78.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-768x200.png 768w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1536x401.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-2048x535.png 2048w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1568x409.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Model Deployment: Traditional Approach vs Syndicai</figcaption></figure>



<p>We also have a tutorial on how to <a href="https://syndicai.co/blog/how-to-deploy-a-pytorch-model-in-minutes">deploy a PyTorch model</a>.</p>



<h3>Prepare a repository with a GPT-2 model </h3>



<p>Apart from putting your model in the GitHub repository, you have to upload two additional files there: <code>requirements.txt</code> and <code>syndicai.py</code>.</p>



<p><code>requirements.txt</code> &#8211; a file with all libraries and frameworks needed to recreate model&#8217;s environment</p>



<pre class="wp-block-code"><code>torch
transformers==2.3.*
wget==3.*</code></pre>



<p><code>syndicai.py</code> &#8211; main file with the <code>PythonPredictor</code> python class responsible for model prediction.</p>



<pre class="wp-block-code python"><code>import wget
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel, GPT2Config
import generator


class PythonPredictor:
    def __init__(self, config):
        medium_config = GPT2Config(n_embd=1024, n_layer=24, n_head=16)
        model = GPT2LMHeadModel(medium_config)
        wget.download(
            "https://convaisharables.blob.core.windows.net/lsp/multiref/medium_ft.pkl",
            "/tmp/medium_ft.pkl",
        )

        weights = torch.load("/tmp/medium_ft.pkl")
        weights&#91;"lm_head.weight"] = weights&#91;"lm_head.decoder.weight"]
        weights.pop("lm_head.decoder.weight", None)

        model.load_state_dict(weights)

        device = "cuda" if torch.cuda.is_available() else "cpu"
        print(f"using device: {device}")
        model.to(device)
        model.eval()

        self.device = device
        self.model = model
        self.tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

    def predict(self, payload):
        conditioned_tokens = self.tokenizer.encode(payload&#91;"text"]) + &#91;generator.END_OF_TEXT]
        prediction = generator.generate(self.model, conditioned_tokens, self.device)
        return self.tokenizer.decode(prediction)</code></pre>



<p>These two files are necessary for the Syndicai tool to be able to recreate the environment and know which function to use for prediction.</p>



<h3>Connect the repository to Syndicai</h3>



<p>When we have the GitHub repository with <code>requirements.txt</code> and <code>syndicai.py</code> ready, we can proceed to connect it to the <a href="https://syndicai.co/">Syndicai</a> platform. In order to that, go to <a href="https://app.syndicai.co/">https://app.syndicai.co/</a>, login, click New Model on the Overview page, and follow the steps in the form. As soon as you finish, the infrastructure will start building. You will need to wait a couple of minutes for the model to become Active.</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/deploy_mode.gif" alt="deploy a model via Syndicai" class="wp-image-256"/><figcaption>Deploy a model via Syndicai Platform</figcaption></figure></div>



<p>For more information about the model preparation or deployment process go to <a href="https://docs.syndic.ai/">Syndicai Docs</a>.</p>



<h2>Step 3: Integrate the model API</h2>



<p>Congratulations!</p>



<p>You now deployed a model on production and have your Rest API ready. To test it out quickly you can paste a sample input script in the Run a model section on <a href="/">Syndicai</a> platform.&nbsp;</p>



<p>Remember that your model needs to be Active in order to work!</p>



<pre class="wp-block-code python"><code>{
    "text": "What is Artificial Intelligence?"
}</code></pre>



<p>If everything works fine, you can now connect the API with any device or service. As an example, you can go to <a href="https://showcase.syndicai.co/">the Showcase page</a> to explore sample implementations.</p>



<p></p>



<h2>Summary</h2>



<p>Today, you have become a slightly better person. You now know how to deploy really cool AI models on production no matter if you are a Data Scientist, Machine Learning Engineer, Backend Developer, DevOps or just enthusiast.&nbsp;</p>



<p>If you found that useful, or you want to get more of those types of tutorials – please drop us a line by mail or catch us on slack.</p>


A quick guide on how to deploy GPT-2 at scale.


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/DDI_deploy_yolo.png" alt="deploy yolov5" class="wp-image-360" title="deploy yolov5" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/DDI_deploy_yolo.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/DDI_deploy_yolo-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/DDI_deploy_yolo-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Part 2 of the two-part series that shows you how to deploy yolov5 which is already trained.</p>



<h2>Introduction</h2>



<p>This is a second part of the series in which we show you how to take your AI startup from zero to one. In the previous blog post, we’ve learned how to <a href="https://syndicai.co/blog/train-yolov5">train yolov5</a> model designed for the Object Detection task. We have a GitHub repository with an AI Model and weights ready.</p>



<p>In this part, we will learn how to deploy YOLOv5 model on production. By the deployment, we mean that you will be able to connect your model to any service and device as a result. Concretely, you will have the REST API connected to a model on a cloud.&nbsp;</p>



<h2>Deployment Process</h2>



<p>The <a href="https://syndicai.co/blog/machine-learning-operations">Machine Learning Operations</a> process usually consists of creating a web service with Flask, recreating the environment in Docker, setting up an infrastructure, and deploying the model to a cloud provider like Google Cloud or AWS.&nbsp;</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1.png" alt="deep learning model deployment" class="wp-image-377" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1-300x78.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267-1-768x200.png 768w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Comparison of AI model deployment between traditional approach and Syndicai approach</figcaption></figure>



<p>Each of these steps requires highly technical knowledge. Even if you succeed after a couple of hours to deploy an AI model, you will most likely find that it will work terribly on production. Your solution won’t be scalable to the increased traffic and the costs of the cloud will not be optimized.&nbsp;&nbsp;&nbsp;</p>



<p>In this tutorial, I will show you how to streamline this process and deploy a PyTorch model with one tool called Syndicai in a few simple clicks.</p>



<h2>Model preparation</h2>



<p>In this section, we will prepare a trained YOLOv5 model to connect it to the Syndicai platform later. In case you missed it, we showed how to train a YOLOv5 model in the previous blog post.</p>



<h4>Weights</h4>



<p>The first thing we will need is the weights of the model. We saved them in the “yolov5s.pt” file. You need to upload them to a service that will allow you to easily and safely download them later using Python code (e.g. Google Drive, Dropbox). For purpose of the tutorial, I will use the weights posted by the authors of the YOLOv5 model: <a aria-label="undefined (opens in a new tab)" href="https://github.com/ultralytics/yolov5/releases/download/v3.1/yolov5s.pt" rel="noreferrer noopener nofollow" target="_blank">https://github.com/ultralytics/yolov5/releases/download/v3.1/yolov5s.pt</a>.&nbsp;</p>



<h4>GitHub repository</h4>



<p>Now, we need to create a GitHub repository with your AI model. Normally, we would start a new project, but since the model is already on GitHub, we can fork the repo using the “Fork” button in the top-right corner. This will copy all the code to our account.&nbsp;</p>



<p>We already have the repository, but we have to spicy it up a little bit so that the Syndicai platform can recognize how it works:</p>



<h5>requirements.txt</h5>



<p>The first file &#8211; <code>requirements.txt</code> &#8211; is already in the forked repository. It contains all the required packages that need to be installed for the model to work.</p>



<pre class="wp-block-code"><code># base ----------------------------------------
Cython
matplotlib>=3.2.2
numpy>=1.18.5
opencv-python==4.2.0.34
Pillow
PyYAML>=5.3
scipy>=1.4.1
tensorboard>=2.2
torch>=1.7.0
torchvision>=0.8.1
tqdm>=4.41.0

# logging -------------------------------------
# wandb

# plotting ------------------------------------
seaborn
pandas

# export --------------------------------------
# coremltools==4.0
# onnx>=1.8.0
# scikit-learn==0.19.2  # for coreml quantization

# extras --------------------------------------
thop  # FLOPS computation
pycocotools>=2.0  # COCO mAP</code></pre>



<h5>syndicai.py</h5>



<p>Next, you will need a file &#8211; <code>syndicai.py</code> &#8211; containing a class <code>PythonPredictor</code> with one function “predict” and constructor. As the name suggests this will instruct the Syndicai platform on how to make a prediction.&nbsp;</p>



<p>The constructor &#8211; <code>__init__</code> &#8211; takes one argument <code>config</code>. The argument is not important for us right now but is necessary nevertheless. The constructor is also the best place to initialize your weights:</p>



<pre class="wp-block-code python"><code>def __init__(self, config):
    urllib.request.urlretrieve("https://github.com/ultralytics/yolov5/releases/download/v3.1/yolov5s.pt", "yolov5s.pt")</code></pre>



<p>Next, let’s take a closer look at the <code>predict</code> function. It has one required argument <code>payload</code> which is a <code>dict</code>. The REST API takes a JSON file which is later transformed into a dict that goes into the “predict” function. </p>



<p>Example input:</p>



<pre class="wp-block-code python"><code>input: {"base64": "R0lGODdhAQABAPAAAP8AAAAAACwAAAAAAQABAAACAkQBADs="}</code></pre>



<p>Since we are operating with images, we can encode them using <code>base64</code> format to send it to the model but we could also input a link to an image.</p>



<p></p>



<p>Regarding the output of the <code>predict</code> function, it should be the </p>



<p>We removed unnecessary parts from the <code>detect.py</code> script from the original repo and left only the code responsible for detecting an object on an image. We display only a part of the code responsible for unpacking input and packing the output. You can find the full code in the repo: <a aria-label="undefined (opens in a new tab)" href="https://github.com/mzmyslowski/yolov5" target="_blank" rel="noreferrer noopener nofollow">https://github.com/mzmyslowski/yolov5</a>.</p>



<pre class="wp-block-code python"><code> def predict(self, payload):
    """ Model Run function """

    im = Image.open(io.BytesIO(base64.b64decode(payload&#91;"base64"])))
    im.save('image.png', 'PNG')

    -- Model specific code --

    img = Image.fromarray(im0)

    im_file = io.BytesIO()
    img.save(im_file, format="PNG")
    im_bytes = base64.b64encode(im_file.getvalue()).decode("utf-8") 

    return im_bytes</code></pre>



<h2>Deployment</h2>



<p>The GitHub repository is ready to be connected to the <a href="https://app.syndicai.co/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Syndicai</a> platform. The only thing to do is to register on the <a href="/">Syndicai</a> website, log in and click “Add model”. You will then need to copy the link to your GitHub repository and name the project. As soon as you finish these steps, the platform will start deploying the model.&nbsp;</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/deploy_mode-1.gif" alt="deploy a deep learning model" class="wp-image-375" title="deploy a deep learning model"/><figcaption>Connect a git repo to deploy a model</figcaption></figure></div>



<p>To monitor the status of the deployment process go to the “Builds” section. You will need to wait a couple of minutes until you see the “Active” status next to your model name.&nbsp;</p>



<p>To test out the integration, go to the “Overview” section and add your input in the form below “Run a model”:</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/run_model-1.gif" alt="run a yolov5 model" class="wp-image-376" title="run a yolov5 model"/><figcaption>Run a REST API request</figcaption></figure></div>



<h2>Conclusion</h2>



<p>In this tutorial, we’ve learned how to deploy yolov5 model fast using the Syndicai platform. We were able to streamline the process that would normally require specialized DevOps. Moreover, the solution is scalable, secure, and cost efficient. This two-part series of articles has allowed us to go from zero to one.</p>


Deploy yolov5 in a few simple clicks


<p></p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/DDI_train_yolo.png" alt="train yolov5" class="wp-image-347" title="train yolov5" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/DDI_train_yolo.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/DDI_train_yolo-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/DDI_train_yolo-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Part 1 of the two-part series that show you how to train yolov5 and later deploy it to production in a simple way.</p>



<h2>Introduction</h2>



<p>In this two-part series, we will go from zero to one in building your AI startup. Well, we will not exactly invent anything new, but we will show you how to train (part 1) and <a href="https://syndicai.co/blog/deploy-yolov5">deploy YOLOv5</a> to production (part 2). By the deployment, we mean that you will be able to connect your model to any service and device as a result.</p>



<p class="has-text-align-left">The problem we will be solving is called Object Detection. For example, I’ve found a beautiful image with a godly creature in the middle on one of my favorites websites &#8211; Unsplash. I would like to know what it is so I could find more images like this on the internet.&nbsp;</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-1024x683.jpg" alt="train yolov5" class="wp-image-339" width="768" height="512" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-1024x683.jpg 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-300x200.jpg 300w, https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-768x512.jpg 768w, https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-1536x1024.jpg 1536w, https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-2048x1365.jpg 2048w, https://data.syndicai.co/wp-content/uploads/2020/12/paul-hanaoka-w2DsS-ZAP4U-unsplash-1568x1045.jpg 1568w" sizes="(max-width: 768px) 100vw, 768px" /><figcaption>Cat picture by Paul Hanaoka (https://unsplash.com/@plhnk)</figcaption></figure></div>



<p>Fortunately, there are already a lot of open-source models that can tackle this problem. We will use the infamous YOLO architecture that achieves state-of-the-art results on many datasets.&nbsp;</p>



<h2>Development&nbsp;</h2>



<p>We will train the <a href="https://github.com/ultralytics/yolov5" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">YOLOv5</a> model which is the newest version of the underlying architecture. We will walk you through the whole process below. For your convenience, we’ve also prepared <a aria-label="undefined (opens in a new tab)" href="https://colab.research.google.com/drive/1URuLxkgkgHMYZQSujerIwhMTVd-pAtRg?usp=sharing" target="_blank" rel="noreferrer noopener nofollow">a Google Colab notebook</a> with all the necessary steps to do it.&nbsp;</p>



<h4>Preparing a picture</h4>



<p>We will first do some imports:</p>



<pre class="wp-block-code python"><code>from IPython.display import Image, clear_output
import requests
import torch</code></pre>



<p>Now, we will download the picture from the Introduction section to the notebook:</p>



<pre class="wp-block-code python"><code>img_data = requests.get('https://unsplash.com/photos/w2DsS-ZAP4U/download?force=true&amp;w=640').content

with open('undentified.jpg', 'wb') as handler:
    handler.write(img_data)
</code></pre>



<p>You can find the image in the “Files” section (the icon looks like a folder) on the left of the screen.</p>



<h4>Cloning GitHub repo</h4>



<p>We will clone the GitHub repo to the notebook:</p>



<pre class="wp-block-code python"><code>!git clone "https://github.com/ultralytics/yolov5.git"</code></pre>



<p>For those that don’t know, the <code>!</code> sign lets you use bash commands inside of the notebook.</p>



<h4>Installing other packages</h4>



<p>We’ve already imported some of the packages, but these are not enough for the model to work. We will install all the required packages using the <code>requiremnets.txt</code> file, but firstly let’s switch the current path to the folder with the model.</p>



<pre class="wp-block-code python"><code>%cd yolov5</code></pre>



<p>Now, we can use <code>pip</code> to install the required packages:</p>



<pre class="wp-block-code python"><code>%pip install -qr requirements.txt</code></pre>



<p>Note that <code>%</code> is a symbol that lets you use the special notebook commands.</p>



<h4>Image dataset</h4>



<p>Before we can train the model, we will need to get an image dataset. We will use a lightweight dataset &#8211; “coco128&#8243; &#8211; recommended by the authors of the model.</p>



<p>The following commands will download the dataset from an official site, unzip it and remove the zip file:</p>



<pre class="wp-block-code python"><code>torch.hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v1.0/coco128.zip', 'tmp.zip')
!unzip -q tmp.zip -d ./ &amp;&amp; rm tmp.zip</code></pre>



<p>Let’s check some image from the dataset:</p>



<pre class="wp-block-code python"><code>Image('coco128/images/train2017/000000000009.jpg', width=600)</code></pre>



<div class="wp-block-image"><figure class="aligncenter"><img src="https://lh4.googleusercontent.com/SOC62ijXjuhtgkw9XzC-nMknmuCRht6-Mxjy9AoC4dFBtj-LNMnyISJodXWoXI5Z84GhLDMf_lfIl6lFixJSbRzrpiPC0547lLw2ldA2TFBDoEw2yPnhj4X6WzQhtrlaH5StvkMV" alt=""/></figure></div>



<p>These things are even harder to identify than our original image! Hopefully, the labelers knew what they were doing.</p>



<h4>Training YOLOv5</h4>



<p>Now, we are ready to train the YOLOv5 model. To do this, we will use the python script provided by the authors. It processes the dataset, trains the model, and logs all of the important information.&nbsp; We will also use some default hyperparameters. If you have time to tune them, you can call the following command to find out more about the settings and hyperparameters.&nbsp;</p>



<pre class="wp-block-code python"><code>!python train.py --help</code></pre>



<p>The command for training to start is</p>



<pre class="wp-block-code python"><code>!python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --nosave --cache</code></pre>



<p>The important thing is that we specified the script to save the weights in the “yolov5s.pt” file. You will need them later when you will be deploying your model.&nbsp;</p>



<h4>Detecting the picture&nbsp;</h4>



<p>Okay, so now the model is trained. Normally, we would examine the performance of the model on the test dataset before deploying it on production, but it’s a matter for another tutorial. Here, we will check what the algorithm can tell us about the picture from the “Introduction” section.</p>



<p>We will use the <code>detect.py</code> script provided by the authors. It will not only detect objects on the image but also add bounding boxes with the names of the object and the probability of them being correct.</p>



<pre class="wp-block-code python"><code>!python detect.py --weights yolov5s.pt --img 640 --conf 0.25 --source ../undentified.jpg
Image(filename='runs/detect/exp/undentified.jpg', width=600)</code></pre>



<div class="wp-block-image"><figure class="aligncenter"><img src="https://lh5.googleusercontent.com/aJuRVKZFE_59YvT-PblLfXqglKiiZX-8GqsPYOvFwPkC-A42kjkmpfeuc281CZkUC2ONkYNvpcWTuheqnCMVqWAt8j3EQ6Nk5FbQqMIMxt0x5xYL7DFbES2aTS8-bu8pIXYAx4GE" alt=""/></figure></div>



<p>It turns out that the godly creature in the middle is a cat! The model gives it 79% confidence. Given the fact that we trained the model for only 3 epochs, it seems like a good result. The model also identified chairs and bowls on the cupboard. Although, the latter ones are not visible well. It’s probably a matter of tweaking the resolution of the picture.&nbsp;&nbsp;&nbsp;</p>



<h2>Conclusion</h2>



<p>In this tutorial, we’ve learned how to train the YOLOv5 model to detect a picture from the Internet. We left out the details that would be typically taken care of by the machine learning engineer. Nevertheless, we familiarised ourselves with the process. We’re in the middle of the zero-to-one path.</p>



<p>In part 2 of the tutorial, we will find out how to <a aria-label="undefined (opens in a new tab)" href="https://syndicai.co/blog/deploy-yolov5" target="_blank" rel="noreferrer noopener">deploy YOLOv5</a> we’ve just trained on production. This is usually a job of DevOps and requires a lot of insider knowledge to do this properly. Nevertheless, we will leverage the power of the <a href="/">Syndicai</a> platform that will allow you to deploy your AI model on production in a few simple clicks.&nbsp;&nbsp;</p>


Train yolov5. A quick guide from a model to the actual use case.


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_keras-1.png" alt="deploy keras model" class="wp-image-431" title="deploy a Keras model in minutes" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_keras-1.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_keras-1-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_keras-1-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Let&#8217;s explore a simple and fast way to deploy a Keras model at scale in minutes without docker and Kubernetes setup. We also have a tutorial on how to <a href="https://syndicai.co/blog/deploy-pytorch-model">deploy a PyTorch model</a>. </p>



<h2>Introduction</h2>



<p>Did you know that almost 90% of Machine Learning courses available on the internet ends up with model training, and do not cover AI model deployment? Models that are trained, but not deployed brings no value to business. However, the whole process is not easy and requires you to possess a deep tech knowledge in this area.</p>



<p>What if there would be an easy and fast way to go from a trained model to an actual use-case with a click of a button. What if that process would be a production-ready one?</p>



<p>You don&#8217;t have to think about it. <a href="/">Syndicai</a> takes care of all the above aspects with no config! Isn&#8217;t it amazing?</p>



<p>In this article, we will show you how to deploy a Keras model in three steps.</p>



<h2>Step 1: Develop a Keras model</h2>



<p>For the purpose of this tutorial, we will go for the Computer Vision task and build a Face Mask Detector. It is one of the hottest models in the last couple of months due to the covid19.</p>



<p>The main goal of that algorithm is to detect and classify whether a person is properly wearing a mask on their face, or not. The whole model is computationally efficient and fast due to <a href="https://arxiv.org/abs/1801.04381v4">MobileNetV2</a> architecture.</p>



<p>It is based on an inverted residual structure where the residual connections are between the bottleneck layers. As a whole, the architecture of MobileNetV2 contains the initial fully convolution layer with 32 filters, followed by 19 residual bottleneck layers.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2021/01/maskmask-1024x395.png" alt="mask face detector" class="wp-image-452" srcset="https://data.syndicai.co/wp-content/uploads/2021/01/maskmask-1024x395.png 1024w, https://data.syndicai.co/wp-content/uploads/2021/01/maskmask-300x116.png 300w, https://data.syndicai.co/wp-content/uploads/2021/01/maskmask-768x296.png 768w, https://data.syndicai.co/wp-content/uploads/2021/01/maskmask.png 1297w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Sample model prediction</figcaption></figure>



<p>We have already prepared the <a href="https://github.com/syndicai/models/tree/marcin/v3/keras/face_mask_detector">Face Mask Detector</a> model on our GitHub repository. You don’t have to do anything with the repository for now.</p>



<p></p>



<h2>Step 2: Deploy a Keras model</h2>



<p>In the traditional workflow of Machine Learning model deployment you need to go through several steps.</p>



<p>Syndicai takes care of all those steps. You just need to connect the git repository with your model and the REST API will be created automatically with one click. Moreover, it takes care of the scalability of resources. The resulting API offers great flexibility because you can connect it to any device.  </p>



<p></p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267.png" alt="" class="wp-image-247" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-300x78.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-768x200.png 768w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1536x401.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-2048x535.png 2048w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1568x409.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Model Deployment: Traditional Approach vs Syndicai</figcaption></figure>



<p>You can also try out to <a href="https://data.syndicai.co/blog/deploy-a-pytorch-model/">deploy a Pytorch model</a>.</p>



<h3>Prepare a repository with a Keras model </h3>



<p>When deploying a model first you need to upload two additional files to your model repository: <code>requirements.txt</code> and <code>syndicai.py</code>.</p>



<p><code>requirements.txt</code> &#8211; a file with all libraries and frameworks needed to recreate model&#8217;s environment</p>



<pre class="wp-block-code"><code>tensorflow==1.15.2
keras==2.3.1
imutils==0.5.3
numpy==1.18.2
opencv-python==4.2.0.*
matplotlib==3.2.1
argparse==1.1
scipy==1.4.1
scikit-learn==0.23.1
pillow==7.2.0</code></pre>



<p><code>syndicai.py</code> &#8211; main file with the <code>PythonPredictor</code> python class responsible for model prediction.</p>



<pre class="wp-block-code python"><code>import os
import cv2
import numpy as np

from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.models import load_model

from utils import url_to_image, b64_to_image, image_to_base64


args = {
    "image": "sample_data/out.jpg",
    "face": "face_detector",
    "model": "model/mask_detector.model",
    "confidence": 0.5,
}


class PythonPredictor:

    def __init__(self, config):
        # load our serialized face detector model from disk
        print("&#91;INFO] loading face detector model...")
        prototxtPath = os.path.sep.join(&#91;args&#91;"face"], "deploy.prototxt"])
        weightsPath = os.path.sep.join(&#91;args&#91;"face"],
                                        "res10_300x300_ssd_iter_140000.caffemodel"])
        self.net = cv2.dnn.readNet(prototxtPath, weightsPath)

        # load the face mask detector model from disk
        print("&#91;INFO] loading face mask detector model...")
        self.model = load_model(args&#91;"model"])

    def predict(self, payload):

        # Get image from url
        try: 
            image = url_to_image(payload&#91;"image_url"])
        except:
            image = b64_to_image(payload&#91;"image_b64"])
        orig = image.copy()
        (h, w) = image.shape&#91;:2]

        # construct a blob from the image
        blob = cv2.dnn.blobFromImage(image, 1.0, (300, 300),
                                     (104.0, 177.0, 123.0))

        # pass the blob through the network and obtain the face detections
        print("&#91;INFO] computing face detections...")
        self.net.setInput(blob)
        detections = self.net.forward()

        # loop over the detections
        for i in range(0, detections.shape&#91;2]):
            # extract the confidence (i.e., probability) associated with
            # the detection
            confidence = detections&#91;0, 0, i, 2]

            # filter out weak detections by ensuring the confidence is
            # greater than the minimum confidence
            if confidence > args&#91;"confidence"]:
                # compute the (x, y)-coordinates of the bounding box for
                # the object
                box = detections&#91;0, 0, i, 3:7] * np.array(&#91;w, h, w, h])
                (startX, startY, endX, endY) = box.astype("int")

                # ensure the bounding boxes fall within the dimensions of
                # the frame
                (startX, startY) = (max(0, startX), max(0, startY))
                (endX, endY) = (min(w - 1, endX), min(h - 1, endY))

                # extract the face ROI, convert it from BGR to RGB channel
                # ordering, resize it to 224x224, and preprocess it
                face = image&#91;startY:endY, startX:endX]
                face = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
                face = cv2.resize(face, (224, 224))
                face = img_to_array(face)
                face = preprocess_input(face)
                face = np.expand_dims(face, axis=0)

                # pass the face through the model to determine if the face
                # has a mask or not
                (mask, withoutMask) = self.model.predict(face)&#91;0]

                # determine the class label and color we'll use to draw
                # the bounding box and text
                label = "Mask" if mask > withoutMask else "No Mask"
                color = (0, 255, 0) if label == "Mask" else (0, 0, 255)

                # include the probability in the label
                label = "{}: {:.2f}%".format(
                    label, max(mask, withoutMask) * 100)

                # display the label and bounding box rectangle on the output
                # frame
                cv2.putText(image, label, (startX, startY - 10),
                                    cv2.FONT_HERSHEY_SIMPLEX, 0.45, color, 2)
                cv2.rectangle(
                    image, (startX, startY), (endX, endY), color, 2)

        return image_to_base64(image)</code></pre>



<p>These two files are necessary for the <a href="https://syndicai.co/">Syndicai</a> tool to be able to recreate the environment and know which function to use for prediction.</p>



<h3>Connect the repository to Syndicai</h3>



<p>When the GitHub repository with all necessary files is ready, we can proceed to connect it to the <a href="https://syndicai.co/">Syndicai</a> platform. </p>



<p>In order to that, go to <a href="https://app.syndicai.co/">https://app.syndicai.co/</a>, login, click New Model on the Overview page, and follow the steps in the form. As soon as you finish, the infrastructure will start building. You will need to wait a couple of minutes for the model to become Active.</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/deploy_mode.gif" alt="" class="wp-image-256"/><figcaption>Deploy a model via Syndicai Platform</figcaption></figure></div>



<p>For more information about the model preparation or deployment process go to <a href="https://docs.syndic.ai/">Syndicai Docs</a>.</p>



<h2>Step 3: Integrate the model API</h2>



<p>You&#8217;ve done it!</p>



<p>Your model is deployed, and your REST API is ready. In order to perform a quick test, just go to copy &amp; paste a sample input script in the Syndicai Platform &#8211; model overview page.</p>



<pre class="wp-block-code python"><code>{
	"image_url": "https://bsmedia.business-standard.com/_media/bs/img/article/2020-07/12/full/1594519569-3012.JPG"
}</code></pre>



<p>Remember that your model needs to be Active in order to work!</p>



<p>If everything works fine, you can now connect the API with any device or service. As an example, you can go to <a href="https://showcase.syndicai.co/">the Showcase page</a> to explore sample implementations.</p>



<p></p>



<h2>Summary</h2>



<p>You had a chance to see how to deploy keras model in minutes. Syndicai allows you to deliver models ready for production without taking care of scalability, monitoring, or security. Just a simple way from the trained model to actual use-case.</p>



<p>If you found that useful, or you want to get more of those types of tutorials – please drop us a line by mail or catch us on slack.</p>


How to deploy Keras model in minutes


<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_pytorch-1.png" alt="deploy pytorch model" class="wp-image-433" title="deploy a PyTorch model in minutes" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/ddi_pytorch-1.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_pytorch-1-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/ddi_pytorch-1-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">In this tutorial, we will cover a simple and fast way to deploy a PyTorch model at scale without a docker, and Kubernetes setup.</p>



<p></p>



<h2 class="has-dark-gray-color has-text-color">I&#8217;ve trained my model. What&#8217;s next?</h2>



<p>Have you ever wanted to deploy a Machine Learning model in order to run it on a website or some mobile app?</p>



<p>In the past, I was exactly in your situation trying different tools, resources. Hundreds of hours spent on building a simple one-page demo app that takes an image as an input and returns one word: Cat. I&#8217;ve experienced that the main focus of most Machine Learning Deployment tutorials ends up with building a simple flask app, creating a docker, and setting up the cloud infrastructure. However, it is still time-consuming and difficult. There are many <a href="https://syndicai.co/blog/problem-with-ai-model-deployment">problems with ai model deployment</a>.</p>



<p>Since that time I&#8217;ve managed to find a way to solve that problem in just three easy steps.</p>



<p>Let&#8217;s have a look and deploy a PyTorch model (Check also <em>How to <a href="https://syndicai.co/blog/deploy-keras-model">deploy keras model</a></em>).</p>



<h2>Step 1: Develop</h2>



<p>In the first step, we need to have a trained model. For this purpose, we will use <strong><a aria-label="undefined (opens in a new tab)" href="https://github.com/ultralytics/yolov5" target="_blank" rel="noreferrer noopener nofollow">a pre-trained PyTorch YoloV5</a></strong>.</p>



<p>For those who don’t know what <a href="https://pjreddie.com/darknet/yolo/" rel="nofollow">Yolo</a> is a real-time object detection framework and stands for <em>You Only Look Once</em>, which means that the image passes only once through the Fully Convolutional Neural Network. Comparing to previous versions the V5 is much faster and much smaller (only 27Mb).</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output.png" alt="" class="wp-image-224" srcset="https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output-300x168.png 300w, https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output-768x429.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /><figcaption>Input of the model (on the left) and output (on the right)</figcaption></figure>



<p>In order to run the algorithm locally, you can copy and paste a script from <a href="https://pytorch.org/hub/ultralytics_yolov5/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">PyTorch Hub</a>. The script will automatically download a pre-trained model and sample images to inference a model.</p>



<pre class="wp-block-code python"><code>import cv2
import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).fuse().autoshape()  # for PIL/cv2/np inputs and NMS

# Images
for f in &#91;'zidane.jpg', 'bus.jpg']:  # download 2 images
    print(f'Downloading {f}...')
    torch.hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v1.0/' + f, f)
img1 = Image.open('zidane.jpg')  # PIL image
img2 = cv2.imread('bus.jpg')&#91;:, :, ::-1]  # OpenCV image (BGR to RGB)
imgs = &#91;img1, img2]  # batched list of images

# Inference
results = model(imgs, size=640)  # includes NMS

# Results
results.show()  # .show() results, .save() jpgs, or .print() to screen</code></pre>



<h2>Step 2: Deploy</h2>



<p>We have our basic script with the model, so now we can deploy it in the cloud.</p>



<p>For this purpose, we will use the <a aria-label="undefined (opens in a new tab)" href="https://app.syndicai.co/" target="_blank" rel="noreferrer noopener">Syndicai Platform</a> which helps developers deliver AI in production effortlessly without setting up an infrastructure. No config needed, no deep tech knowledge required. <a href="https://syndicai.co/">Syndicai</a> takes care of security, scalability, monitoring, and more. You just need to prepare a model and connect your git repository. The model will be deployed automatically.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267.png" alt="" class="wp-image-247" srcset="https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1024x267.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-300x78.png 300w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-768x200.png 768w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1536x401.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-2048x535.png 2048w, https://data.syndicai.co/wp-content/uploads/2020/12/traditional_syndicai-1568x409.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Model Deployment: Traditional Approach vs Syndicai</figcaption></figure>



<h3>Prepare a model</h3>



<p>During the model preparation process you need to ensure the following files are in the git repository:</p>



<p>1st file: <code>requirements.txt</code> &#8211; file with all libraries and frameworks needed to recreate model&#8217;s environment</p>



<pre class="wp-block-code text"><code>opencv-python==4.2.0.34
pillow
pyyaml
tqdm
torch
matplotlib
torchvision
scipy</code></pre>



<p>2nd file: <code>syndicai.py</code> &#8211; main file with the <code>PythonPredictor</code> python class responsible for model prediction.</p>



<pre class="wp-block-code python"><code>import torch
from PIL import Image 
from helpers import draw_box, url_to_img, img_to_bytes


class PythonPredictor:

    def __init__(self, config):
        """ Download pretrained model. """
        self.model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()

    def predict(self, payload):
        """ Run a model based on url input. """

        # Inference
        img = url_to_img(payload&#91;"url"])
        results = self.model(img)

        # Draw boxes
        boxes = results.xyxy&#91;0].numpy()
        box_img = draw_box(img, boxes)

        # Save image
        #box_img.save("sample_data/output.png", "PNG")

        return img_to_bytes(box_img)</code></pre>



<p>Those files are necessary to correctly recreate your environment, build the infrastructure, and finally run the model in the form of a webservice. You can check with the <a href="https://github.com/syndicai/models/tree/master/pytorch/yolov5" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Syndicai model repository.</a></p>



<h3>Connect a repository</h3>



<p>When the model is ready it&#8217;s time for deployment. The only thing that you need to do at this stage is to log in to <a href="https://app.syndicai.co" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Syndicai Platform</a>, click <strong>New Model</strong> on the Overview page, and follow the steps in the form. After the model is deployed the whole magic happens in order to give you the model <strong>API</strong> in the Integrate section.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/upload_model.gif" alt="" class="wp-image-225" width="829" height="438"/><figcaption>Deploy a Model by connecting the repository.</figcaption></figure></div>



<p>For more information about the model preparation or deployment process go to <a href="https://docs.syndic.ai/">Syndicai Docs</a>.</p>



<h2>Step 3: Integrate</h2>



<p>Great!</p>



<p>We have our scalable API so theoretically the job is done. You can run a model on the Platform or integrate the API with your app.</p>



<p>In order to make an inference the model go to Syndicai platform and paste a sample input script in the <strong>Run a model</strong> section</p>



<pre class="wp-block-code python"><code>{
    "url": "https://images.pexels.com/photos/2083866/pexels-photo-2083866.jpeg?auto=compress&amp;cs=tinysrgb&amp;h=750&amp;w=1260"
}</code></pre>



<p>Remember that your model needs to be Active in order to work!</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/run_model.gif" alt="" class="wp-image-232" width="800" height="423"/><figcaption>Run a model on the Platform</figcaption></figure></div>



<p>API of the model can be integrated with some React app or a website in order to build <a href="https://showcase.syndicai.co/" target="_blank" rel="noreferrer noopener">a Showcase</a>. </p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/run_showcase.gif" alt="" class="wp-image-228" width="805" height="426"/><figcaption>Integrate a model with Syndicai Showcase page.</figcaption></figure></div>



<p>Fork the repository with <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/showcase" target="_blank" rel="noreferrer noopener">the Showcase</a> and explore yolov5 implementation on your own.</p>



<h2>Summary</h2>



<p>The main goal of that tutorial was to easily deploy the PyTorch model in production. No matter what is your specialty, whether you are a Data Scientist, a Machine Learning Engineer, Backend Engineer, or DevOps. Now you are able to deliver an AI model without any configuration and infrastructure setup.</p>



<p>You can also explore <em>How to <a href="https://syndicai.co/blog/deploy-keras-model">deploy Keras model</a></em>. </p>



<p>If you found that useful, or you want to get more of those types of tutorials &#8211; please drop us a line by <a href="mailto:hello@syndicai.co">mail</a> or catch us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw">slack</a>.</p>


How to deploy a PyTorch model in minutes


<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/09/05_ML_monitoring.png" alt="Machine Learning Monitoring in Production" class="wp-image-202" srcset="https://data.syndicai.co/wp-content/uploads/2020/09/05_ML_monitoring.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/09/05_ML_monitoring-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/09/05_ML_monitoring-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Machine Learning Monitoring is a very important aspect of the whole MLOps workflow which tells us about model availability and performance.</p>



<p></p>



<h2>Why Machine Learning Monitoring is important?</h2>



<p>In traditional software, we plan every single step and operation in order to reduce as much as possible the risk of error in the production environment. We can do it using automated tests to ensure that our applications or system perform as expected.</p>



<p>When it comes to Machine Learning the situation is significantly different. A trained model provided by a Data Scientists is nothing else than just a representation of historical data or events. Such a model deployed in a production environment is constantly bombarded with a new type of information which sometimes causes undesirable behavior.</p>



<p>Since we can’t explicitly test for all of the possible machine learning prediction cases, we need to continuously monitor that system to ensure it’s operating effectively. Moreover, such monitoring needs to be performed in a specific way and based on special requirements.</p>



<p></p>



<h2>What is Machine Learning Monitoring?</h2>



<p>It is important to understand what is happening with models in the production environment. Monitoring is a very important part of the whole <a href="https://syndicai.co/blog/machine-learning-operations">Machine Learning Operations</a> workflow. However, is not an already defined term. Some people consider monitoring as a model availability, while others look at this from a model performance perspective. Both of them are very important, especially when we talk about automation.</p>



<h4>Is the model available?</h4>



<p>When we talk about the availability we mainly mean resource utilization. We are checking whether the model has enough resources? How it scales during high frequency of incoming requests? What is the consumption of CPU, GPU and memory?</p>



<h4>How is model performing?</h4>



<p>In terms of model performance, our goal is to find information about traffic patterns, latency, or errors. We mainly ask what is the accuracy, what is the drift? When is it time to retrain?</p>



<h2>What are Machine Learning Monitoring metrics?</h2>



<p>When it comes to metrics there are many ways how you can do it. However, the most important question to rise is: What you want to measure? What is the most important factor for you?</p>



<p>In terms of Model performance the most popular metrics are: </p>



<ul><li>Type 1 Error</li><li>Type 2 Error </li><li>Accuracy</li><li>Precision</li><li>Recall</li><li>F1 score</li><li>Mean Absolute Error (MAE)</li><li>Mean Squared Error (MSE)</li></ul>



<p></p>


What is Machine Learning Monitoring in Production?


<figure class="wp-block-image size-large"><img src="http://data.syndic.ai/wp-content/uploads/2020/09/04_mlops.png" alt="what is MLOps" class="wp-image-195" srcset="https://data.syndicai.co/wp-content/uploads/2020/09/04_mlops.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/09/04_mlops-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/09/04_mlops-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /><figcaption>What is Machine Learning Operations &#8211; MLOps</figcaption></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Why we need to talk about Machine Learning Operations, and how it is different from DevOps.</p>



<h2>Introduction</h2>



<p>When we talk about Machine Learning we always have data and the model in our mind. All the challenges correlated to the <a href="https://towardsdatascience.com/exploratory-data-analysis-8fc1cb20fd15" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Exploratory Data Analysis</a> workflow. However, there is a big missing puzzle in the whole game &#8211; Deployment. In order to extract real business value from an ML model, you need to somehow put it into production. And this &#8216;somehow&#8217; is a methodology called Machine Learning Operations.</p>



<h2>Why we need to talk about MLOps?</h2>



<p>We can hear a lot of great news about the new Machine Learning models, techniques, and tools. Everybody can agree that the industry in this field is moving forward extremely quickly. However, there is a big bottleneck in AI. Taking models and moving them into production where they actually start to solve real problems is a big challenge. There are a lot of <a href="https://syndicai.co/blog/problems-with-ai-model-deployment">problems with AI model deployment</a>. Just like DevOps did 15 years ago taking off the ground a piece of code with just a single code repository, the whole AI ecosystem fights to achieve a similar outcome with ML models. Execute a build pipeline that takes a model to production along the way testing and validating every single step. Sounds easy but it&#8217;s not.</p>



<h2>What the term &#8220;Machine Learning Operations&#8221; really means?</h2>



<p>MLOps itself is not something that is already defined. Both in terms of the definition (companies mean slightly different things) and the people who should deal with it in the company (ML Team or Backend Team). The whole topic is still fresh and the terminology has not yet been defined and established.</p>



<p>In <a href="/">syndicai</a> when we talk about MLOps, we talk about the ability to move AI models from data scientist&#8217;s machines or laptops to remote machine clusters at scale. The main goal of that process is to automate the whole workflow taking care of resource management, orchestration, data/model versioning, and monitoring.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/09/EDA_MLOps_graph-1024x501.png" alt="" class="wp-image-199" srcset="https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-1024x501.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-300x147.png 300w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-768x376.png 768w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-1536x752.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-2048x1003.png 2048w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-1568x768.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Exploratory Data Analysis (EDA) is a fraction of Machine Learning Operations (MLOps) workflow</figcaption></figure>



<h2>How MLOps is different from DevOps?</h2>



<p>The main idea behind DevOps is that a piece of software is tested, QA-ed by the development team when stable goes into production.</p>



<p>In the field of AI, everything around is actually very different.</p>



<h4><strong>Experimental Environment</strong> </h4>



<p>Unlike DevOps, the MLOps is much more experimental at its core. During the whole workflow, Machine Learning engineers try different features, parameters, model architectures. It is important not only to version the whole code but also being able to reproduce previously obtained results.</p>



<h4><strong>Performance degradation</strong> </h4>



<p>When we deploy our models, the constant flow of new data causes the decrease of accuracy over time, which is not the case with the regular software. That is why we need to retrain and deploy again models that theoretically work properly (with no bugs) but have a significant drop of the performance.</p>



<h4><strong>Resource management</strong></h4>



<p>When it comes to computing resources, we need to be aware that Machine learning experiments can be very heavy workloads. So unlike regular software, there is a need to run stuff on large machines from day one. Very often there is a need to change workstations depending on the traffic.</p>



<h4><strong>Model monitoring</strong></h4>



<p>In the DevOps environment mainly infrastructure is under constant supervision. In MLOps there is a need to take care of the model itself constantly checking what happens with parameters and metrics in order to retrain when needed.</p>



<h4><strong>Testing</strong></h4>



<p>In addition to the software tests such as unit testing and integration testing there is a need for model validation, model training tests.</p>



<h2>Conclusion</h2>



<p>The awareness of Machine Learning operations is constantly growing. More and more companies are providing their services on top of AI which requires them to automate processes. We can also feel that the whole movement empowers the industry bringing to life new tools. Everything in order to become much more clever and efficient as a Data Scientist.</p>



<p>We need to be aware that unlike &#8216;normal software&#8217;, Machine Learning models need to be treated with a different set of practices and tools. It is all about implementing Continuous integration flow into Data Science workflow. </p>


What exactly is Machine Learning Operations?

<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/07/04_algorithms_worth_to_follow.png" alt="" class="wp-image-146" srcset="https://data.syndicai.co/wp-content/uploads/2020/07/04_algorithms_worth_to_follow.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/07/04_algorithms_worth_to_follow-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/07/04_algorithms_worth_to_follow-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure></div>



<p class="has-text-color has-large-font-size has-light-gray-color">Each year new deep learning algorithms are faster, smarter, and lighter than their younger brothers. Explore how to focus on the most promising ones, and which ones are worth exploring.</p>



<h2>Executive summary</h2>



<p>Hundreds of new deep learning algorithms are presented each year. Developers and engineers working in the field of AI are constantly bombarded with models that are faster, smarter, and lighter than their younger brothers. How to focus on the most promising ones, and which ones are worth exploring? In this article, we will share our position which one you should follow or try.</p>



<p></p>







<p></p>



<h2>YoloV5</h2>



<h4>Description</h4>



<p>The first algorithm worth exploring is one of the most popular in the field of Computer Vision &#8211; YOLO (&#8220;You Only Look Once&#8221;). It is a number one choice when it comes to Real-time object detection and classification algorithms. The main idea behind Yolo algorithm is to create features from input images. Later feed these features through a prediction system to draw boxes around objects and finally predict their classes. Yolov5 <a href="https://towardsdatascience.com/yolov5-compared-to-faster-rcnn-who-wins-a771cd6c9fb4" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">outperforms</a> not only previous Yolo versions but also <a href="https://towardsdatascience.com/faster-r-cnn-for-object-detection-a-technical-summary-474c5b857b46" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">FasterRCNN</a>. Model runs about 2.5 times faster and is more accurate detecting smaller objects. The results are also cleaner with little to no overlapping boxes.</p>



<h4>Potential applications</h4>



<ul><li><a href="https://medium.com/analytics-vidhya/retail-store-item-detection-using-yolov5-7ba3ddd71b0c" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Retail store object detection</a></li><li><a href="https://easychair.org/publications/preprint/RlQ2" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Car Damage Classification and Detection</a></li><li><a href="https://www.nature.com/articles/s41598-019-50795-9" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Whale counting in satellite and aerial images</a></li></ul>



<h4>Materials</h4>



<ul><li>Demo: <a href="https://showcase.syndicai.co/yolov5">Try it out</a></li><li>Code: <a href="https://github.com/syndicai/models">YoloV5 implementation</a></li><li>Article: <a href="https://blog.netcetera.com/object-detection-and-tracking-in-2020-f10fb6ff9af3" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Object detection and tracking 2020</a></li></ul>



<p></p>



<h2>Super resolution</h2>



<h4>Description</h4>



<p>When looking for a picture for the presentation we quickly find the appropriate one. However, many of these photos are really low resolution. The solution to that problem might be Super-Resolution algorithm. It learns new details pixel by pixel increasing resolution, sharpening, and upscaling the image without losing its content. The whole <a href="https://igl.ethz.ch/projects/prosr/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">algorithm</a> is based on <a href="https://medium.com/sigmoid/a-brief-introduction-to-gans-and-how-to-code-them-2620ee465c30" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Generative Adversarial Networks (GANs)</a>. Upsampling an image to increase its resolution to twice its original size takes less than a second. We can go up to eight times the original resolution.</p>



<h4>Potential applications</h4>



<ul><li>Image enhancing in <a href="https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">professional image editing</a> software</li><li><a href="https://www.youtube.com/watch?v=CKoHF2Wtmgw" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Upscale old film footage</a></li><li>Super-resolution for <a href="https://www.earthcube.eu/post/marie-saintraymond-earthcube-eu-2" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">satellite imagery analysis</a></li></ul>



<h4>Materials</h4>



<ul><li>Demo: Try it out</li><li>Papers + Code: <a href="https://paperswithcode.com/task/image-super-resolution" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Image Super Resolution</a></li><li>Article: <a href="https://medium.com/towards-artificial-intelligence/reading-esrgan-enhanced-super-resolution-generative-adversarial-networks-super-resolution-e8533ad006b5" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">ESRGAN — Enhanced Super-Resolution GAN</a></li></ul>



<p></p>



<h2>Image Inpainting</h2>



<h4>Description</h4>



<p>According to <a href="https://en.wikipedia.org/wiki/Inpainting" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Wikipedia</a> &#8216;Inpainting is a conservation process where damaged, deteriorating, or missing parts of an artwork are filled in to present a complete image. This process can be applied to both physical and digital art mediums such as oil or acrylic paintings, chemical photographic prints, 3-dimensional sculptures, or digital images and video&#8217;. </p>



<p>In the field of AI, <a href="https://medium.com/sigmoid/a-brief-introduction-to-gans-and-how-to-code-them-2620ee465c30" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Generative adversarial networks</a> are typically used for this sort of implementation, given their ability to “generate” new data, or in this case, the missing information. </p>



<h4>Potential applications</h4>



<ul><li><a href="https://www.youtube.com/watch?v=gg0F5JjKmhA" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Damaged photographs and video restoration</a></li><li>Text removal</li><li>Watermarks &amp; Entire object removal</li></ul>



<h4>Materials</h4>



<ul><li>Article: <a href="https://medium.com/jamieai/image-inpainting-with-deep-learning-dd8555e56a32" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Image Inpainting with Deep Learning</a></li><li>Article: <a href="https://heartbeat.fritz.ai/guide-to-image-inpainting-using-machine-learning-to-edit-and-correct-defects-in-photos-3c1b0e13bbd0" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Guide to Image inpainting</a></li><li>Papers: <a href="https://paperswithcode.com/task/image-inpainting" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">Image Inpainting</a></li></ul>



<p></p>



<h2>Closing Remarks</h2>



<p>Thank to AI you can either color the black-and-white images, generate faces, or even translate lip motion to text. Most of those algorithms are based on Generative Adversarial Networks which is the most powerful Neural Network architecture nowadays. </p>



<p>If you find that article helpful or you are interested in running some amazing project with us, join us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw">slack</a> or <a href="mailto:hello@syndicai.co">drop us a line</a>! </p>



<p>Currently, we are looking for AI Ambassadors!</p>



<p></p>

deep learning algorithms to follow in 2020

Deep Learning algorithms worth to follow in 2020


<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/07/ai_nextgrid_seed.jpg" alt="AI seed program" class="wp-image-80" title="AI seed program" srcset="https://data.syndicai.co/wp-content/uploads/2020/07/ai_nextgrid_seed.jpg 1000w, https://data.syndicai.co/wp-content/uploads/2020/07/ai_nextgrid_seed-300x169.jpg 300w, https://data.syndicai.co/wp-content/uploads/2020/07/ai_nextgrid_seed-768x432.jpg 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Your goal is not to sell a product but to inspire with your vision, a big vision because if you want to play big you need to act big &#8211; a little bit about AI seed program.</p>



<p></p>



<h2 id="9b0b"><strong>Executive summary</strong></h2>



<p><strong>3 months of super-exciting, super-intense program where you’re pushed to the boundaries of your own creativity, time management, and persuasion in order to deliver next-generation product crafted to disrupt.</strong> I think that’s the best one-sentence-summary of what the Nextgrid AI seed program is about. If you’re curious how the program looked like, why you should try it or not try it, and who is the best fit — feel invited to the following paragraphs.</p>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/1497/1*FIULCkKYFiW2_n-1o8qorg.png" alt="Mission of the Nextgrid accelerator" title="Mission of the Nextgrid accelerator"/><figcaption>Mission of the Nextgrid accelerator</figcaption></figure>



<h2 id="3f6e">What is this all about?</h2>



<p>The main idea of the AI seed program is to gather wise minds specialized in AI and help them monetize their visions. It consists of three phases — <strong>plan</strong>, <strong>build</strong>, and <strong>communicate</strong>, each one created in order to develop one of your most important entrepreneurial skills.</p>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/1222/1*YagNODRQyYNCu5qVMtfz2Q.png" alt="Three phases of the three-month program" title="Three phases of the three-month program"/><figcaption>Three phases of the three-month program</figcaption></figure>



<p>The first phase was mainly about the roadmap creation. We had to create a tech and business strategy for the project in order to assure that the <a href="https://medium.com/agileactors/the-minimum-viable-product-explained-8f1187ca7cec" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">MVP</a> will be delivered on the final day. We learned not only how to perform customer validation but also learned a lot about project execution and goal setting.</p>



<p>The second phase was mainly a technical one and our job was to focus on product development. Equipped with credits for the cloud and a lot of guidance from tech mentors working in the field of AI, product design, and development we were able to keep a high pace and stay focused during that demanding time.</p>



<p>The third phase was dedicated to preparation for the final demo day, where we had to present our startup in front of Venture Capital investors and companies like Amazon, Google, Intel, IBM, and Nvidia. During that time we had a chance to participate in workshops prepared by specialists working in the field of communication, marketing strategy, and acting. We have done a lot of training sessions and preparation. Everything in order to minimize stress and make that final pitch the best one we could ever give.</p>



<h2 id="8e31"><strong>Ok, so what I learned?</strong></h2>



<h3 id="2f60"><strong>1. Be ‘stubborn on vision and flexible in approach’</strong></h3>



<p>Jeff Bezos quote exactly explains what kind of mindset you should have not only when attending a program but first of all when starting a business. It’s super important to have a clear vision of what exactly is the problem you are trying to solve and what would be your desired outcome if that would work. If you stick too much to your idea or solution, and not the customer’s pain point — you will find yourself in the quagmire of unprofitable business.</p>



<h3 id="65cb">2. Ask proper questions</h3>



<p>One of the biggest mistakes I‘ve done in the past was getting opinions about the product, not people’s past experience, the problems they faced, and the tools they have tried. It’s like asking your mother for an opinion about your first drawing of a cat that looks like a machine gun — she will always tell you that’s the best ever piece of art she has ever seen. To overcome that situation we might go for the simple <a aria-label="undefined (opens in a new tab)" href="https://www.youtube.com/watch?v=MT4Ig2uqjTc" target="_blank" rel="noreferrer noopener nofollow">5-question guide</a> introduced by Eric Migicovsky which is a very good starting point for a lot of founders.</p>



<h3 id="8919">3. Become a listener</h3>



<p>This might sounds a little bit odd, because theoretically founders should be great speakers not listeners. However, if you want to be a good leader first you need to be good listener. The difference between hearing and listening is that hearing is understanding words that are coming to you, listening is trying to get to the meaning of the words that are said to you. When you talk to customers it is important to extract information even if something was not said directly or was missed during the conversation.</p>



<h3 id="7296">4. Plan and execute with OKR’s</h3>



<p>Tech-savvy entrepreneurs might face a problem with the project execution and product on-time delivery because they always want to build everything from scratch. A perfect solution might be a concept intruduced by Google which is called OKR (Objectives and Key Results). It is a management and execution technic that set strategy and goals over a specified amount of time for an organization and teams. Objectives are any initiatives that you hope to accomplish, while Key Results are numerically-based expressions of success or progress towards an Objective.</p>



<h3 id="0731">5. <strong>Keep pace and focus on what really matters</strong></h3>



<p>Three months&#8217; time is a very short period of time to bring some really good value. However, a startup is the type of environment where you will always face lack of time so it is important to focus on things that really matters. To choose which step should be the next it’s good to ask yourself <em>Which next activity will give you the biggest step forward towards a desired goal? </em>That might be solving some blockers or making some improvements in the product.</p>



<h2 id="168e">Conclusion</h2>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/1920/1*vOp_SBi7FuF5ovXsomtkkg.jpeg" alt="Nobody said that you can’t become Tony Stark of the next generation." title="Nobody said that you can’t become Tony Stark of the next generation."/><figcaption>Nobody said that you can’t become Tony Stark of the next generation.</figcaption></figure>



<p>It’s important to have in mind that during three months AI seed program even the greatest program will never make you an entrepreneurial star. However, right community and wise people can show you the right direction and give tools which might help you significantly in the process of making your vision come true, so it’s important to make the right choice choosing a program.</p>



<p>I hope that this grasp of information inspired you a little bit to make a change and pursue your ideas. Everything I learned and experienced allowed me to push my project to a whole new level giving a lot of energy and believe in what’s next. It was a “smashing” program full of intense work that I specially recommend it to those who already created some sort of the first AI concept and now want to get out of the garage in order to share their great idea with the world.</p>



<p>Everybody is saying <em>be yourself</em>, and I will tell you <em>become someone</em>.</p>



<h2 id="05fd">Closing remarks</h2>



<p>Last but not least you might ask me ‘…so what is a project you have been working on?’</p>



<blockquote class="wp-block-quote"><p>As we all know AI is the major driving force in many applications we see today. However, in order to put that AI in those applications, we need to go through a certain process. That process starts with <strong>data</strong> gathering and preparation, goes through <strong>model</strong> building, and training, and finish with <strong>AI model deployment, and integration</strong>. When it comes to the first two steps — it’s not a problem anymore for developers and engineers. The problem is with the last step — deployment &amp; integration…</p><p>…so what’s the solution? The solution is <a aria-label="undefined (opens in a new tab)" href="/" target="_blank" rel="noreferrer noopener">syndicai</a>.</p></blockquote>


AI seed program by Nextgrid, what I learned?


<figure class="wp-block-image size-large"><img src="http://data.syndic.ai/wp-content/uploads/2020/06/deployment_model.png" alt="problems with ai model deployment" class="wp-image-41" title="problems with ai model deployment" srcset="https://data.syndicai.co/wp-content/uploads/2020/06/deployment_model.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/06/deployment_model-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/06/deployment_model-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Short overview of problems with AI model deployment and pain points moving Deep Learning models into production environment.</p>



<p></p>



<h2>Executive Summary</h2>



<p>I spoke with almost fifty AI specialists working in different industries. Everything in order to get their perspective on problems with AI model deployment in production. Many treasured insights encouraged me to write the following article, so I hope you enjoy it.</p>



<p></p>



<h2>Why?</h2>



<p>Two years ago I finished my Master @&nbsp;Imperial College London. I had a chance to be a part of&nbsp;BICI Lab working on advanced deep learning algorithms for healthcare. Getting plausible final results I decided to share my work with business people in the form of a sample app where people could test the solution using their data, not just read the paper (which was hard to understand even for people in the field of AI). I realized that it is not an easy task and even for big companies it is an issue.</p>



<p>Since that time I decided to pursue the passion of fast and easy AI model deployment. Investigating different solutions and approaches I tried to reach people dealing with this topic daily.</p>



<p></p>



<h2>What is AI deployment?</h2>



<p>Some of you might ask what exactly is AI model deployment? To clarify, “AI model deployment” is a process that starts when a model is trained and ends up with a model implemented in some mobile app or a webpage. Behind this process, there is a lot of iterative and very often arduous work.</p>



<p>Google engineers have pointed out in one of their&nbsp;<a aria-label="undefined (opens in a new tab)" href="https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf" target="_blank" rel="noreferrer noopener nofollow">papers</a>&nbsp;that the ML code in only a small fraction of the whole ecosystem. The process of deployment plays a big role. It consists of many components like resource management, system monitoring, versioning, serving infrastructure.</p>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/2000/1*JzopLqICdPyx6-GJYTV2TQ.png" alt="ai model deployment workfow"/><figcaption>Only a small fraction of real-world ML systems are composed of the ML code. The process of deployment is practically 50% of it (right side).</figcaption></figure>



<p></p>



<h2>Well, so what is the hardest part of the process of model deployment?</h2>



<p>From the architecture point of view one of the biggest problems that AI engineers face nowadays is the&nbsp;<strong>maintenance&nbsp;</strong>of models running in the production environment. Collecting events about behavior is one thing but extracting valuable information from it is still a tricky part. Sometimes it is hard to keep models up to date with business assumptions.</p>



<p>Since Machine Learning workflow is a fast-paced environment,&nbsp;<strong>versioning</strong>&nbsp;of models and datasets causes a lot of effort. Especially in a situation when we need to restore some previous configuration. Keeping up to date all versions clearly and understandably is a key to perform fast actions.</p>



<p>When it comes to algorithms, the next hard part is model&nbsp;<strong>optimization</strong>. Many popular Machine Learning and Data Science frameworks, like sci-kit-learn are simply not prepared for the production environment. Engineers need to spend a lot of time understanding the code created by scientists and optimize each part according to requirements.</p>



<p>It is also important to mention that most of those frameworks are written in python in which multithreading is not multithreading, due to the GIL (for heavy tech details please visit the&nbsp;<a href="https://hackernoon.com/concurrent-programming-in-python-is-not-what-you-think-it-is-b6439c3f3e6a" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">following article</a>). This problem is called <strong>parallelism</strong>. All those models very often end up rewritten to C language or some hardware-specific language, only weights left the same.</p>



<p>Going further into hardware there is a problem with&nbsp;<strong>hardware compatibility</strong>&nbsp;when we want to convert our model from a specific framework to edge or mobile. Lack of hardware-specific libraries slows down product delivery sometimes even for a couple of weeks.</p>



<p>Last but not least, from the business point of view there is a lack of <a href="https://syndicai.co/blog/machine-learning-operations">Machine Learning Operations</a> (MLOps). The people who stand between the Machine Learning team and operational engineers. People able to manage the production lifecycle and understand distributed systems. Because Machine Learning is emerging in practically all areas of our lives there will be growing demand for Engineers who specialize in Machine-Learning-To-Production areas.</p>



<p></p>



<h2>What don’t you love about the solutions you’ve tried?</h2>



<p>The cost structure provided by many platforms is not clear and understandable. It is very hard to&nbsp;<strong>predict and plan expenditure</strong>. Sometimes even simple estimation causes a lot of problems, which causes many platforms to lose customers.</p>



<p>The more options have a platform, the better documented it should be to give a full overview of all possibilities and advantages over the competition. However, here we also have a gap —&nbsp;<strong>a weak code documentation</strong>. There is lack of tutorials, examples, and potential use cases with a short explanation. </p>



<p>On the market, there is a&nbsp;<strong>lack of platforms</strong>&nbsp;able to deploy models for mobile applications, where hardware specification plays an important role, since models are running on the device not on the cloud. Many platforms provide a lot of tools for general model deployment, however, only a couple of them are domain-specific platforms e.g for like healthcare or energy sector.</p>



<p>Popular optimization-for-hardware libraries can handle only a small fraction of popular frameworks, which is inconvenient when we deal with big infrastructures and models.</p>



<p>It is also important that switching from one platform to another also plays a big role. We have to always keep in mind&nbsp;<em>how much time it will take to find a new tool</em>&nbsp;vs&nbsp;<em>how much time it will take to transfer the current solution to the new tool</em>&nbsp;vs&nbsp;<em>how much time do we have to the project deadline</em>&nbsp;vs&nbsp;<em>whether the new tool has all features we need</em>.</p>



<p>All those remarks play an important role. However, we have to remember that at the end of the day for the customer the most important thing is the fact that it works. Speed and efficiency are no guarantee of success so it’s important to deliver.</p>



<p></p>



<h2>Problem or a challenge?</h2>



<p>To sum up we might ask,&nbsp;<em>Is the whole AI model deployment process&nbsp;</em><strong><em>a problem</em></strong><em>&nbsp;— a matter of doubt and uncertainty, or&nbsp;</em><strong><em>a challenge</em></strong><em>&nbsp;— an opportunity for success and growth?</em></p>



<p>In my opinion, it is a challenge worth exploring more in detail. My final thoughts are as follows.</p>



<p>Currently, there is a small concern about who should be responsible for deployments and later maintenance. In most cases, ML Engineers / AI developers are not responsible for AI model deployment. Backend engineers are the ones who deal with it.</p>



<p>Future platforms have to be transparent in costs and able to version the model. There is a need for the software able to optimize models for specific hardware e.g. mobiles, edge, or websites.</p>



<p>Finally, led by inspirational Leonardo’s da Vinci quote&nbsp;<em>“simplicity is the ultimate sophistication”</em>&nbsp;I need to admit that there is also a place for simple and easy-to-use tools centered around one particular field of industry or task allowing fast prototyping and testing.</p>



<p>That is what inspired us to build <a href="/">Syndicai</a> &#8211; a platform that solves problems with ai model deployment.</p>


artificial intelligence deployment - a problem or a challenge

AI model deployment, a problem or a challenge?


<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/06/marin.png" alt="" class="wp-image-36" srcset="https://data.syndicai.co/wp-content/uploads/2020/06/marin.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/06/marin-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/06/marin-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">On the mission to shape the future of ai deployment with Syndicai Platform.</p>



<p></p>



<h2 id="9bf8"><strong>Executive summary</strong></h2>



<p>AI model deployment is one of the biggest challenges for Machine Learning engineers nowadays. It&#8217;s a very hard and time-consuming process that requires particular deep tech knowledge in specific areas. In&nbsp;<a aria-label="undefined (opens in a new tab)" href="/" target="_blank" rel="noreferrer noopener">syndicai</a>&nbsp;we are on a mission to change shaping the future of AI deployment. We are building&nbsp;<strong>a platform that aims to simplify the deployment and integration of AI models in production. </strong>&nbsp;We believe that making AI more accessible and usable will allow engineers to focus on solving big problems more efficiently. Rather than spending time on setting up the infrastructure.</p>



<h2><strong>Problem</strong></h2>



<p>In order to build a machine learning model, we need to collect, clean, and prepare data, as well as train, test, and tune a model. If everything goes according to plan and the model is able to perform the desired task with high accuracy we’re done. Unfortunately&nbsp;<strong>for business people, trained but not deployed model does not bring any value</strong>&nbsp;to a company. Despite the fact that this last step is very important, only a&nbsp;<a href="https://www.forbes.com/sites/cognitiveworld/2020/03/31/modelops-is-the-key-to-enterprise-ai/#7e98863a6f5a" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener nofollow">small percentage</a>&nbsp;of AI models reach that final phase. There are&nbsp;several reasons&nbsp;for that, the main ones are as follows:</p>



<ol><li><strong>Computing<br></strong>Deep learning algorithms require a lot of computing power not only for training but also for model inference. Clever management of those resources is a big challenge and it’s not easy.</li><li><strong>Scalability<br></strong>Some libraries and frameworks used by Data Scientists are not adapted to perform distributed operations. </li><li><strong>Lack of resources to learn<br></strong>Roughly 95% of AI courses on the internet covers only the first part of the Machine Learning workflow finishing with the trained model</li></ol>



<p>A list of <a href="https://syndicai.co/blog/problem-with-ai-model-deployment">problems with AI model deployment</a> is quite long.</p>



<h2><strong>Solution</strong></h2>



<p>We believe in a clever way of putting AI models into production that smoothen the whole process. As true lovers of simplicity and advanced automation, we see a huge opportunity in this area. We feel challenged to face that.</p>



<p>Here comes the time to introduce you to our solution to the problem — <a href="https://syndicai.co/" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">syndicai</a> platform.</p>



<blockquote class="wp-block-quote"><p>Syndicai is on a mission to simplify the process of making AI models accessible and usable to AI developers and engineers. By cutting out the middleman and creating a platform that is so easy to use that even your grandmother can use it.</p></blockquote>



<p>We want every developer (especially ML engineer) to be able to deploy and implement a trained AI model. Without dealing with resource management, scalability, computation, and security.</p>



<h2><strong>Product</strong></h2>



<p>With our platform we took as our main focus in the area of AI model deployment and implementation. Our goal is to make this process smooth and fast.</p>



<blockquote class="wp-block-quote"><p>No middleman, no deep tech knowledge required — you get a simple way from model to actual use-case.</p></blockquote>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/4000/1*4RNekv7TDJvsgYjPPx4_Pw.jpeg" alt="syndicai AI deployment &amp; implementation workflow"/><figcaption>Drag &amp; Drop your AI model to deploy and implement using various frameworks and libraries — syndicai will take care of everything whats in the middle.</figcaption></figure>



<h3><strong>Step 1: Deploy</strong></h3>



<p>In order to deploy a model, first, you need to&nbsp;<a aria-label="undefined (opens in a new tab)" href="https://docs.syndic.ai/getting-started/quick-start" target="_blank" rel="noreferrer noopener">place two additional files&nbsp;</a>in your model directory:&nbsp;<strong>syndicai.py</strong>&nbsp;which contains only python function and&nbsp;<strong>requirements.txt</strong>. Those files are necessary to recreate the working environment and allow to access the model by web service. When it’s done you just&nbsp;<strong>drag &amp; drop your trained model on the platform.&nbsp;</strong>The whole process takes roughly 20min.</p>



<h3><strong>Step 2: Implement</strong></h3>



<p>Developers who want to implement a model get specially generated code snippet ready-made to copy and paste in different environments.</p>



<h2><strong>Closing remarks</strong></h2>



<p>Our vision is to change the way developers implement AI models into production by allowing them to easily share their work and focus on their proficiency instead of building infrastructure.</p>



<p>Currently, we are at the stage of testing our alpha version of the platform, so if you feel inspired and curious about the upcoming stages or would you like to test the platform before launching — feel free to join us on&nbsp;<a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" rel="noreferrer noopener">slack</a>&nbsp;or leave us&nbsp;<a href="https://syndic.ai/" target="_blank" rel="noreferrer noopener">an email</a>.</p>


Blog

The latest news about AI in production from the Syndicai team.