
<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/11/ddi_pytorch_in_minutes.png" alt="" class="wp-image-208" srcset="https://data.syndicai.co/wp-content/uploads/2020/11/ddi_pytorch_in_minutes.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/11/ddi_pytorch_in_minutes-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/11/ddi_pytorch_in_minutes-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">In this tutorial, we will cover a simple and fast way to deploy a PyTorch model in production using the Syndicai Platform.</p>



<p></p>



<h2 class="has-dark-gray-color has-text-color">I&#8217;ve trained my model. What&#8217;s next?</h2>



<p>Have you ever wanted to deploy a Machine Learning model in order to run it on a website or some mobile app?</p>



<p>In the past, I was exactly in your situation trying different tools, resources. Hundreds of hours spent on building a simple one-page demo app that takes an image as an input and returns one word: Cat. I&#8217;ve experienced that the main focus of most Machine Learning Deployment tutorials ends up with building a simple flask app, creating a docker, and setting up the cloud infrastructure. However, it is still time-consuming and difficult.</p>



<p>Since that time I&#8217;ve managed to find a way to solve that problem in just three easy steps.</p>



<p>Let&#8217;s have a look.</p>



<h2>Step 1: Develop</h2>



<p>In the first step, we need to have a trained model. For this purpose, we will use <strong><a aria-label="undefined (opens in a new tab)" href="https://github.com/ultralytics/yolov5" target="_blank" rel="noreferrer noopener nofollow">a pre-trained PyTorch YoloV5</a></strong>.</p>



<p>For those who don’t know what <a href="https://pjreddie.com/darknet/yolo/" rel="nofollow">Yolo</a> is a real-time object detection framework and stands for <em>You Only Look Once</em>, which means that the image passes only once through the Fully Convolutional Neural Network. Comparing to previous versions the V5 is much faster and much smaller (only 27Mb).</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output.png" alt="" class="wp-image-224" srcset="https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output-300x168.png 300w, https://data.syndicai.co/wp-content/uploads/2020/11/inpupt_output-768x429.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /><figcaption>Input of the model (on the left) and output (on the right)</figcaption></figure>



<p>In order to run the algorithm locally, you can copy and paste a script from <a href="https://pytorch.org/hub/ultralytics_yolov5/" rel="nofollow">PyTorch Hub</a>. The script will automatically download a pre-trained model and sample images to inference a model.</p>



<pre class="wp-block-code"><code>import cv2
import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).fuse().autoshape()  # for PIL/cv2/np inputs and NMS

# Images
for f in &#91;'zidane.jpg', 'bus.jpg']:  # download 2 images
    print(f'Downloading {f}...')
    torch.hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v1.0/' + f, f)
img1 = Image.open('zidane.jpg')  # PIL image
img2 = cv2.imread('bus.jpg')&#91;:, :, ::-1]  # OpenCV image (BGR to RGB)
imgs = &#91;img1, img2]  # batched list of images

# Inference
results = model(imgs, size=640)  # includes NMS

# Results
results.show()  # .show() results, .save() jpgs, or .print() to screen</code></pre>



<h2>Step 2: Deploy</h2>



<p>We have our basic script with the model, so now we can deploy it in the cloud.</p>



<p>For this purpose, we will use the <a aria-label="undefined (opens in a new tab)" href="https://app.syndicai.co/" target="_blank" rel="noreferrer noopener">Syndicai Platform</a> which helps developers deliver AI in production effortlessly without setting up an infrastructure. No config needed, no deep tech knowledge required. <a href="https://syndicai.co/">Syndicai</a> takes care of security, scalability, monitoring, and more. You just need to prepare a model and connect your git repository. The model will be deployed automatically.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-1024x267.png" alt="" class="wp-image-221" srcset="https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-1024x267.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-300x78.png 300w, https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-768x200.png 768w, https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-1536x401.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-2048x534.png 2048w, https://data.syndicai.co/wp-content/uploads/2020/11/workflow-2-1568x409.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>AI Model Deployment: Traditional Approach vs Syndicai</figcaption></figure>



<h3>Prepare a model</h3>



<p>During the model preparation process you need to ensure the following files are in the git repository:</p>



<ol><li><code>requirements.txt</code> &#8211; file with all libraries and frameworks needed to recreate model&#8217;s environment</li><li><code>syndicai.py</code> &#8211; main file with the <code>syndicai</code> python class which is responsible for model prediction.</li></ol>



<p>Those files are necessary to correctly recreate your environment, build the infrastructure, and finally run the model in the form of a webservice. You can check with the <a href="https://github.com/syndicai/models/tree/master/pytorch/yolov5" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Syndicai model repository.</a></p>



<h3>Connect a repository</h3>



<p>When the model is ready it&#8217;s time for deployment. The only thing that you need to do at this stage is to log in to <a href="https://app.syndicai.co">Syndicai Platform</a>, click <strong>New Model</strong> on the Overview page, and follow the steps in the form. After the model is deployed the whole magic happens in order to give you the model <strong>API</strong> in the Integrate section.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/upload_model.gif" alt="" class="wp-image-225" width="829" height="438"/><figcaption>Deploy a Model by connecting the repository.</figcaption></figure></div>



<p>For more information about the model preparation or deployment process go to <a href="https://docs.syndic.ai/">Syndicai Docs</a>.</p>



<h2>Step 3: Integrate</h2>



<p>Great!</p>



<p>We have our scalable API so theoretically the job is done. You can run a model on the Platform or integrate the API with your app.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/run_model.gif" alt="" class="wp-image-232" width="800" height="423"/><figcaption>Run a model on the Platform</figcaption></figure></div>



<p>However, some of you might be still not satisfied since the pure API doesn’t solve the problem. Syndicai currently works on <a href="https://github.com/syndicai/showcase">a Web app boilerplate</a> that can automatically generate <a aria-label="undefined (opens in a new tab)" href="https://showcase.syndicai.co/" target="_blank" rel="noreferrer noopener">a Showcase</a> based on your model API.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://data.syndicai.co/wp-content/uploads/2020/11/run_showcase.gif" alt="" class="wp-image-228" width="805" height="426"/><figcaption>Integrate a model with Syndicai Showcase page.</figcaption></figure></div>



<p>You can fork the repository with <a aria-label="undefined (opens in a new tab)" href="https://github.com/syndicai/showcase" target="_blank" rel="noreferrer noopener">the Showcase</a> and explore yolov5 implementation.</p>



<h2>Summary</h2>



<p>The main goal of that tutorial was to easily deploy the PyTorch model in production. No matter what is your specialty, whether you are a Data Scientist, a Machine Learning Engineer, Backend Engineer, or DevOps. Now you are able to deliver an AI model without any configuration and infrastructure setup.</p>



<p>If you found that useful, or you want to get more of those types of tutorials &#8211; please drop us a line by <a href="mailto:hello@syndicai.co">mail</a> or catch us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw">slack</a>.</p>


How to deploy a PyTorch model in minutes


<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/09/05_ML_monitoring.png" alt="Machine Learning Monitoring in Production" class="wp-image-202" srcset="https://data.syndicai.co/wp-content/uploads/2020/09/05_ML_monitoring.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/09/05_ML_monitoring-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/09/05_ML_monitoring-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Machine Learning Monitoring is a very important aspect of the whole MLOPs workflow which tells us about model availability and performance.</p>



<p></p>



<h2>Why Machine Learning Monitoring is important?</h2>



<p>In traditional software, we plan every single step and operation in order to reduce as much as possible the risk of error in the production environment. We can do it using automated tests to ensure that our applications or system perform as expected.</p>



<p>When it comes to Machine Learning the situation is significantly different. A trained model provided by a Data Scientists is nothing else than just a representation of historical data or events. Such a model deployed in a production environment is constantly bombarded with a new type of information which sometimes causes undesirable behavior.</p>



<p>Since we can’t explicitly test for all of the possible machine learning prediction cases, we need to continuously monitor that system to ensure it’s operating effectively. Moreover, such monitoring needs to be performed in a specific way and based on special requirements.</p>



<p></p>



<h2>What is Machine Learning Monitoring?</h2>



<p>It is important to understand what is happening with models in the production environment. Monitoring is a very important part of the whole <a href="https://syndic.ai/a/what-exactly-is-machine-learning-operations">Machine Learning Operations workflow</a>. However, is not already defined term. Some people consider monitoring as a model availability, while others look at this from a model performance perspective. Both of them are very important, especially when we talk about automation.</p>



<h4>Is the model available?</h4>



<p>When we talk about the availability we mainly mean resource utilization. We are checking whether the model has enough resources? How it scales during high frequency of incoming requests? What is the consumption of CPU, GPU and memory?</p>



<h4>How is model performing?</h4>



<p>In terms of model performance, our goal is to find information about traffic patterns, latency, or errors. We mainly ask what is the accuracy, what is the drift? When is it time to retrain?</p>



<h2>What are Machine Learning Monitoring metrics?</h2>



<p>When it comes to metrics there are many ways how you can do it. However, the most important question to rise is: What you want to measure? What is the most important factor for you?</p>



<p>In terms of Model performance the most popular metrics are: </p>



<ul><li>Type 1 Error</li><li>Type 2 Error </li><li>Accuracy</li><li>Precision</li><li>Recall</li><li>F1 score</li><li>Mean Absolute Error (MAE)</li><li>Mean Squared Error (MSE)</li></ul>



<p></p>


What is Machine Learning Monitoring in Production?


<figure class="wp-block-image size-large"><img src="http://data.syndic.ai/wp-content/uploads/2020/09/04_mlops.png" alt="what is MLOps" class="wp-image-195" srcset="https://data.syndicai.co/wp-content/uploads/2020/09/04_mlops.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/09/04_mlops-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/09/04_mlops-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /><figcaption>What is Machine Learning Operations &#8211; MLOps</figcaption></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Why we need to talk about Machine Learning Operations, what it is exactly, and how it is different from DevOps.</p>



<h2>Introduction</h2>



<p>When we talk about Machine Learning we always have data and the model in our mind. All the challenges correlated to the <a href="https://towardsdatascience.com/exploratory-data-analysis-8fc1cb20fd15" rel="nofollow">Exploratory Data Analysis</a> workflow. However, there is a big missing puzzle in the whole game &#8211; Deployment. In order to extract real business value from an ML model, you need to somehow put it into production. And this &#8216;somehow&#8217; is a methodology called Machine Learning Operations.</p>



<h2>Why we need to talk about MLOps?</h2>



<p>We can hear a lot of great news about the new Machine Learning models, techniques, and tools. Everybody can agree that the industry in this field is moving forward extremely quickly. However, there is a big bottleneck AI. Taking models and moving them into production where they actually are starting to solve real problems is <a href="/a/ai-deployment-problem-or-challenge" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">a big challenge</a>. Just like DevOps did 15 years ago taking off the ground a piece of code with just a single code repository, the whole AI ecosystem fights to achieve a similar outcome with ML models. Execute a build pipeline that takes a model to production along the way testing and validating every single step. Sounds easy but it&#8217;s not.</p>



<h2>What the term &#8220;Machine Learning Operations&#8221; really means?</h2>



<p>MLOps itself is not something that is already defined. Both in terms of the definition (companies mean slightly different things) and the people who should deal with it in the company (ML Team or Backend Team). The whole topic is still fresh and the terminology has not yet been defined and established.</p>



<p>In <a href="https://syndic.ai/">syndicai</a> when we talk about MLOps, we talk about the ability to move AI models from data scientist&#8217;s machines or laptops to remote machine clusters at scale. The main goal of that process is to automate the whole workflow taking care of resource management, orchestration, data/model versioning and monitoring.</p>



<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/09/EDA_MLOps_graph-1024x501.png" alt="" class="wp-image-199" srcset="https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-1024x501.png 1024w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-300x147.png 300w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-768x376.png 768w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-1536x752.png 1536w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-2048x1003.png 2048w, https://data.syndicai.co/wp-content/uploads/2020/09/EDA_MLOps_graph-1568x768.png 1568w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Exploratory Data Analysis (EDA) is a fraction of Machine Learning Operations (MLOps) workflow</figcaption></figure>



<h2>How MLOps is different from DevOps?</h2>



<p>The main idea behind DevOps is that a piece of software is tested, QA-ed by the development team when stable goes into production.</p>



<p>In the field of AI, everything around is actually very different.</p>



<h4><strong>Experimental Environment</strong> </h4>



<p>Unlike DevOps, the MLOps is much more experimental at its core. During the whole workflow, Machine Learning engineers try different features, parameters, model architectures. It is important not only to version the whole code but also being able to reproduce previously obtained results.</p>



<h4><strong>Performance degradation</strong> </h4>



<p>When we deploy our models, the constant flow of new data causes the decrease of accuracy over time, which is not the case with the regular software. That is why we need to retrain and deploy again models that theoretically work properly (with no bugs) but have a significant drop of the performance.</p>



<h4><strong>Resource management</strong></h4>



<p>When it comes to computing resources, we need to be aware that Machine learning experiments can be very heavy workloads. So unlike regular software, there is a need to run stuff on large machines from day one. Very often there is a need to change workstations depending on the traffic.</p>



<h4><strong>Model monitoring</strong></h4>



<p>In the DevOps environment mainly infrastructure is under constant supervision. In MLOps there is a need to take care of the model itself constantly checking what happens with parameters and metrics in order to retrain when needed.</p>



<h4><strong>Testing</strong></h4>



<p>In addition to the software tests such as unit testing and integration testing there is a need for model validation, model training tests.</p>



<h2>Conclusion</h2>



<p>The awareness of Machine Learning operations is constantly growing. More and more companies are providing their services on top of AI which requires them to automate processes. We can also feel that the whole movement empowers the industry bringing to life new tools. Everything in order to become much more clever and efficient as a Data Scientist.</p>



<p>We need to be aware that unlike &#8216;normal software&#8217;, Machine Learning models need to be treated with a different set of practices and tools. It is all about implementing Continuous integration flow into Data Science workflow. </p>


What exactly is Machine Learning Operations?

<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/07/04_algorithms_worth_to_follow.png" alt="" class="wp-image-146" srcset="https://data.syndicai.co/wp-content/uploads/2020/07/04_algorithms_worth_to_follow.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/07/04_algorithms_worth_to_follow-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/07/04_algorithms_worth_to_follow-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure></div>



<p class="has-text-color has-large-font-size has-light-gray-color"></p>



<h2>Executive summary</h2>



<p>Hundreds of new algorithms are presented each year. Developers and engineers working in the field of AI are constantly bombarded with models that are faster, smarter, and lighter than their younger brothers. How to focus on the most promising ones, and which ones are worth exploring? In the following article, we will share our position which one you should follow or try.</p>



<p></p>







<p></p>



<h2>YoloV5</h2>



<h4>Description</h4>



<p>YOLO (&#8220;You Only Look Once&#8221;) is one of the most popular algorithms in the field of Computer Vision. Number one choice when it comes to Real-time object detection and classification algorithms. The main idea behind Yolo algorithm is to create features from input images. Later feed these features through a prediction system to draw boxes around objects and finally predict their classes. Yolov5 <a aria-label="undefined (opens in a new tab)" href="https://towardsdatascience.com/yolov5-compared-to-faster-rcnn-who-wins-a771cd6c9fb4" target="_blank" rel="noreferrer noopener">outperforms</a> not only previous Yolo versions but also <a href="https://towardsdatascience.com/faster-r-cnn-for-object-detection-a-technical-summary-474c5b857b46">FasterRCNN</a>. Model runs about 2.5 times faster and is more accurate detecting smaller objects. The results are also cleaner with little to no overlapping boxes.</p>



<h4>Potential applications</h4>



<ul><li><a aria-label="undefined (opens in a new tab)" href="https://medium.com/analytics-vidhya/retail-store-item-detection-using-yolov5-7ba3ddd71b0c" target="_blank" rel="noreferrer noopener">Retail store object detection</a></li><li><a href="https://easychair.org/publications/preprint/RlQ2" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Car Damage Classification and Detection</a></li><li><a href="https://www.nature.com/articles/s41598-019-50795-9" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Whale counting in satellite and aerial images</a></li></ul>



<h4>Materials</h4>



<ul><li>Demo: <a href="/showcase/yolo">Try it out</a></li><li>Code: <a href="https://github.com/syndicai/sample-models/tree/master/yolo_v5">YoloV5 implementation</a></li><li>Article: <a aria-label="undefined (opens in a new tab)" href="https://blog.netcetera.com/object-detection-and-tracking-in-2020-f10fb6ff9af3" target="_blank" rel="noreferrer noopener">Object detection and tracking 2020</a></li></ul>



<p></p>



<h2>Super resolution</h2>



<h4>Description</h4>



<p>When looking for a picture for the presentation we quickly find the appropriate one. However, many of these photos are really low resolution. The solution to that problem might be Super-Resolution algorithm. It learns new details pixel by pixel increasing resolution, sharpening, and upscaling the image without losing its content. The whole <a aria-label="undefined (opens in a new tab)" href="https://igl.ethz.ch/projects/prosr/" target="_blank" rel="noreferrer noopener">algorithm</a> is based on <a aria-label="undefined (opens in a new tab)" href="https://medium.com/sigmoid/a-brief-introduction-to-gans-and-how-to-code-them-2620ee465c30" target="_blank" rel="noreferrer noopener">Generative Adversarial Networks (GANs)</a>. Upsampling an image to increase its resolution to twice its original size takes less than a second. We can go up to eight times the original resolution.</p>



<h4>Potential applications</h4>



<ul><li>Image enhancing in <a aria-label="undefined (opens in a new tab)" href="https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/" target="_blank" rel="noreferrer noopener">professional image editing</a> software</li><li><a aria-label="undefined (opens in a new tab)" href="https://www.youtube.com/watch?v=CKoHF2Wtmgw" target="_blank" rel="noreferrer noopener">Upscale old film footage</a></li><li>Super-resolution for <a aria-label="undefined (opens in a new tab)" href="https://www.earthcube.eu/post/marie-saintraymond-earthcube-eu-2" target="_blank" rel="noreferrer noopener">satellite imagery analysis</a></li></ul>



<h4>Materials</h4>



<ul><li>Demo: <a href="/showcase/super" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Try it out</a></li><li>Papers + Code: <a aria-label="undefined (opens in a new tab)" href="https://paperswithcode.com/task/image-super-resolution" target="_blank" rel="noreferrer noopener">Image Super Resolution</a></li><li>Article: <a aria-label="undefined (opens in a new tab)" href="https://medium.com/towards-artificial-intelligence/reading-esrgan-enhanced-super-resolution-generative-adversarial-networks-super-resolution-e8533ad006b5" target="_blank" rel="noreferrer noopener">ESRGAN — Enhanced Super-Resolution GAN</a></li></ul>



<p></p>



<h2>Image Inpainting</h2>



<h4>Description</h4>



<p>According to <a aria-label="undefined (opens in a new tab)" href="https://en.wikipedia.org/wiki/Inpainting" target="_blank" rel="noreferrer noopener">Wikipedia</a> &#8216;Inpainting is a conservation process where damaged, deteriorating, or missing parts of an artwork are filled in to present a complete image. This process can be applied to both physical and digital art mediums such as oil or acrylic paintings, chemical photographic prints, 3-dimensional sculptures, or digital images and video&#8217;. </p>



<p>In the field of AI, <a href="https://medium.com/sigmoid/a-brief-introduction-to-gans-and-how-to-code-them-2620ee465c30" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">Generative adversarial networks</a> are typically used for this sort of implementation, given their ability to “generate” new data, or in this case, the missing information. </p>



<h4>Potential applications</h4>



<ul><li><a aria-label="undefined (opens in a new tab)" href="https://www.youtube.com/watch?v=gg0F5JjKmhA" target="_blank" rel="noreferrer noopener">Damaged photographs and video restoration</a></li><li>Text removal</li><li>Watermarks &amp; Entire object removal</li></ul>



<h4>Materials</h4>



<ul><li>Article: <a aria-label="undefined (opens in a new tab)" href="https://medium.com/jamieai/image-inpainting-with-deep-learning-dd8555e56a32" target="_blank" rel="noreferrer noopener">Image Inpainting with Deep Learning</a></li><li>Article: <a aria-label="undefined (opens in a new tab)" href="https://heartbeat.fritz.ai/guide-to-image-inpainting-using-machine-learning-to-edit-and-correct-defects-in-photos-3c1b0e13bbd0" target="_blank" rel="noreferrer noopener">Guide to Image inpainting</a></li><li>Papers: <a aria-label="undefined (opens in a new tab)" href="https://paperswithcode.com/task/image-inpainting" target="_blank" rel="noreferrer noopener">Image Inpainting</a></li></ul>



<p></p>



<h2>Closing Remarks</h2>



<p>Thank to AI you can either color the black-and-white images, generate faces, or even translate lip motion to text. Most of those algorithms are based on Generative Adversarial Networks which is the most powerful Neural Network architecture nowadays. </p>



<p>If you find that article helpful or you are interested in running some amazing project with us, join us on <a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw">slack</a> or <a href="mailto:hello@syndic.ai">drop us a line</a>! </p>



<p>Currently, we are looking for AI Ambassadors!</p>



<p></p>

deep learning algorithms to follow in 2020

Deep Learning algorithms worth to follow in 2020


<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/07/ai_nextgrid_seed.jpg" alt="" class="wp-image-80" srcset="https://data.syndicai.co/wp-content/uploads/2020/07/ai_nextgrid_seed.jpg 1000w, https://data.syndicai.co/wp-content/uploads/2020/07/ai_nextgrid_seed-300x169.jpg 300w, https://data.syndicai.co/wp-content/uploads/2020/07/ai_nextgrid_seed-768x432.jpg 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Your goal is not to sell a product but to inspire with your vision, a big vision because if you want to play big you need to act big.</p>



<p></p>



<h2 id="9b0b"><strong>Executive summary</strong></h2>



<p><strong>3 months of super-exciting, super-intense program where you’re pushed to boundaries of your own creativity, time management, and persuasion in order to deliver next-generation product crafted to disrupt.</strong> I think that’s the best one-sentence-summary of what <a href="https://nextgrid.ai/seed/" target="_blank" rel="noreferrer noopener">Nextgrid AI Seed Program</a> is about. If you’re curious how the program looked like, why you should try it or not try it, and who is the best fit — feel invited to the following paragraphs.</p>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/1497/1*FIULCkKYFiW2_n-1o8qorg.png" alt="Image for post"/><figcaption>Mission of the Nextgrid accelerator</figcaption></figure>



<h2 id="3f6e">What is this all about?</h2>



<p>The main idea of the program is to gather wise minds specialized in AI and help them monetize their visions. It consists of three phases — <strong>plan</strong>, <strong>build</strong>, and <strong>communicate</strong>, each one created in order to develop one of your most important entrepreneurial skills.</p>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/1222/1*YagNODRQyYNCu5qVMtfz2Q.png" alt="Image for post"/><figcaption>Three phases of the three-month program</figcaption></figure>



<p>During the first phase, our goal was to create a tech and business roadmap for the project in order to assure that the <a target="_blank" rel="noreferrer noopener" href="https://medium.com/agileactors/the-minimum-viable-product-explained-8f1187ca7cec">MVP</a> will be delivered on the final day. We learned not only how to perform customer validation but also learned a lot about project execution and goal setting.</p>



<p>The second phase was mainly a technical one and our job was to focus on product development. Equipped with credits for the cloud and a lot of guidance from tech mentors working in the field of AI, product design, and development we were able to keep a high pace and stay focused during that demanding time.</p>



<p>The third phase was dedicated to preparation for the final demo day, where we had to present our startup in front of Venture Capital investors and companies like Amazon, Google, Intel, IBM and Nvidia. During that time we had a chance to participate in workshops prepared by specialists working in the field of communication, marketing strategy and acting. A lot of training sessions and preparation was done. Everything in order to minimize stress and make that final pitch the best one we could ever give.</p>



<h2 id="8e31"><strong>Ok, so what I learned?</strong></h2>



<h3 id="2f60"><strong>1. Be ‘stubborn on vision and flexible in approach’</strong></h3>



<p>Jeff Bezos quote exactly explains what kind of mindset you should have not only when attending a program but first of all when starting a business. It’s super important to have a clear vision of what exactly is the problem you are trying to solve and what would be your desired outcome if that would work. If you stick too much to your idea or solution, and not the customer’s pain point — you will find yourself in the quagmire of unprofitable business.</p>



<h3 id="65cb">2. Ask proper questions</h3>



<p>One of the biggest mistakes I‘ve done in the past was getting opinion about the product, not people’s past experience, problems they faced, and the tools they have tried. It’s like asking your mother for an opinion about your first drawing of a cat which looks like a machine gun — she will always tell you that’s the best ever piece of art she has ever seen. To overcome that situation we might go for the simple <a href="https://www.youtube.com/watch?v=MT4Ig2uqjTc" target="_blank" rel="noreferrer noopener">5-question guide</a> introduced by Eric Migicovsky which is a very good starting point for a lot of founders.</p>



<h3 id="8919">3. Become a listener</h3>



<p>This might sounds a little bit odd, because theoretically founders should be great speakers not listeners. However, if you want to be a good leader first you need to be good listener. The difference between hearing and listening is that hearing is understanding words that are coming to you, listening is trying to get to the meaning of the words that are said to you. When you talk to customers it is important to extract information even if something was not said directly or was missed during the conversation.</p>



<h3 id="7296">4. Plan and execute with OKR’s</h3>



<p>Tech-savvy entrepreneurs might face a problem with the project execution and product on-time delivery, because they always want to build everything from scratch. A perfect solution to that might be a concept used by Google which is called OKR (Objectives and Key Results). It is a management and execution technic that set strategy and goals over a specified amount of time for an organization and teams. Objectives are any initiatives that you hope to accomplish, while Key Results are numerically-based expressions of success or progress towards an Objective.</p>



<h3 id="0731">5. <strong>Keep pace and focus on what really matters</strong></h3>



<p>Three months&#8217; time is a very short period of time to bring some really good value. However, a startup is the type of environment where you will always face lack of time so it is important to focus on things that really matters. To choose which step should be the next it’s good to ask yourself <em>Which next activity will give you the biggest step forward towards a desired goal? </em>That might be solving some blockers or making some improvements in the product.</p>



<h2 id="168e">Conclusion</h2>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/1920/1*vOp_SBi7FuF5ovXsomtkkg.jpeg" alt="Image for post"/><figcaption>Nobody said that you can’t become Tony Stark of the next generation.</figcaption></figure>



<p>It’s important to have in mind that during three months even the greatest program will never make you a entrepreneurial star. However, right community and wise people can show you the right direction and give tools which might help you significantly in the process of making your vision come true, so it’s important to make a right choice choosing a program.</p>



<p>I hope that this grasp of information inspired you a little bit to make a change and pursue your ideas. Everything I learned and experienced allowed me to push my project to a whole new level giving a lot of energy and believe in what’s next. It was a “smashing” program full of intense work that I specially recommend it to those who already created some sort of the first AI concept and now want to get out of the garage in order to share their great idea with the world.</p>



<p>Everybody is saying <em>be yourself</em>, and I will tell you <em>become someone</em>.</p>



<h2 id="05fd">Closing remarks</h2>



<p>Last but not least you might ask me ‘…so what is a project you have been working on?’</p>



<blockquote class="wp-block-quote"><p>As we all know AI is the major driving force in many applications we see today. However, in order to put that AI in those applications, we need to go through a certain process. That process starts with <strong>data</strong> gathering and preparation, goes through <strong>model</strong> building, and training, and finish with <strong>deployment, and integration</strong>. When it comes to first two steps — it’s not a problem anymore for developers and engineers. The problem is with last step — deployment &amp; integration…</p><p>…so what’s the solution? The solution is <a href="https://syndic.ai/" target="_blank" rel="noreferrer noopener">syndicai</a>.</p></blockquote>


What I learned from the AI Nextgrid Seed Program?


<figure class="wp-block-image size-large"><img src="http://data.syndic.ai/wp-content/uploads/2020/06/deployment_model.png" alt="artificial intelligence deployment - a problem or a challenge" class="wp-image-41" srcset="https://data.syndicai.co/wp-content/uploads/2020/06/deployment_model.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/06/deployment_model-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/06/deployment_model-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">Short overview of pain points moving Deep Learning models into production environment.</p>



<p></p>



<h2><strong>Executive Summary</strong></h2>



<p>About 50 conversations with AI specialists working in different industries share their position about obstacles with AI models in production. Many treasured insights encouraged me to write the following article, so I hope you enjoy it.</p>



<p></p>



<h2><strong>Why?</strong></h2>



<p>Two years ago I finished my Master @&nbsp;<a href="https://www.imperial.ac.uk/" target="_blank" rel="noreferrer noopener">Imperial College London</a>. I had a chance to be a part of&nbsp;<a aria-label="undefined (opens in a new tab)" href="http://www.bici-lab.org/" target="_blank" rel="noreferrer noopener">BICI</a> Lab working on advanced deep learning algorithms for healthcare. Getting plausible final results I decided to share my work with business people in the form of a sample app where people could test the solution using their data, not just read the paper (which was hard to understand even for people in the field of AI). I realized that it is not an easy task and even for big companies it is an issue.</p>



<p>Since that time I decided to pursue the passion of fast and easy AI model deployment. Investigating different solutions and approaches I tried to reach people dealing with this topic daily.</p>



<p></p>



<h2><strong>What is AI deployment?</strong></h2>



<p>Some of you might ask what exactly is AI model deployment? To clarify, “AI model deployment” is a process that starts when a model is trained and ends up with a model implemented in some mobile app or a webpage. Behind this process, there is a lot of iterative and very often arduous work.</p>



<p>Google engineers have pointed out in one of their&nbsp;<a href="https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf" target="_blank" rel="noreferrer noopener">papers</a>&nbsp;that the ML code in only a small fraction of the whole ecosystem. The process of deployment plays a big role. It consists of many components like resource management, system monitoring, versioning, serving infrastructure.</p>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/2000/1*JzopLqICdPyx6-GJYTV2TQ.png" alt=""/><figcaption>Only a small fraction of real-world ML systems are composed of the ML code. The process of deployment is practically 50% of it (right side).</figcaption></figure>



<p></p>



<h2><strong>Well, so what is the hardest part of the process of model deployment?</strong></h2>



<p>From the architecture point of view one of the biggest problems that AI engineers face nowadays is the&nbsp;<strong>maintenance&nbsp;</strong>of models running in the production environment. Collecting events about behavior is one thing but extracting valuable information from it is still a tricky part. Sometimes it is hard to keep models up to date with business assumptions.</p>



<p>Since Machine Learning workflow is a fast-paced environment,&nbsp;<strong>versioning</strong>&nbsp;of models and datasets causes a lot of effort. Especially in a situation when we need to restore some previous configuration. Keeping up to date all versions clearly and understandably is a key to perform fast actions.</p>



<p>When it comes to algorithms, the next hard part is model&nbsp;<strong>optimization</strong>. Many popular Machine Learning and Data Science frameworks, like sci-kit-learn are simply not prepared for the production environment. Engineers need to spend a lot of time understanding the code created by scientists and optimize each part according to requirements.</p>



<p>It is also important to mention that most of those frameworks are written in python in which multithreading is not multithreading, due to the GIL (for heavy tech details please visit the&nbsp;<a href="https://hackernoon.com/concurrent-programming-in-python-is-not-what-you-think-it-is-b6439c3f3e6a" target="_blank" rel="noreferrer noopener">following article</a>). This problem is called <strong>parallelism</strong>. All those models very often end up rewritten to C language or some hardware-specific language, only weights left the same.</p>



<p>Going further into hardware there is a problem with&nbsp;<strong>hardware compatibility</strong>&nbsp;when we want to convert our model from a specific framework to edge or mobile. Lack of hardware-specific libraries slows down product delivery sometimes even for a couple of weeks.</p>



<p>Last but not least, from the business point of view there is a lack of <a href="https://towardsdatascience.com/ml-ops-machine-learning-as-an-engineering-discipline-b86ca4874a3f" target="_blank" rel="noreferrer noopener">MLOps</a>. The people who stand between Machine Learning team and operational engineers. People able to manage the production lifecycle and understand distributed systems. Because Machine Learning is emerging in practically all areas of our lives there will be growing demand for Engineers who specialize in Machine-Learning-To-Production areas.</p>



<p></p>



<h2><strong>What don’t you love about the solutions you’ve tried?</strong></h2>



<p>The cost structure provided by many platforms is not clear and understandable. It is very hard to&nbsp;<strong>predict and plan expenditure</strong>. Sometimes even simple estimation causes a lot of problems, which causes many platforms to lose customers.</p>



<p>The more options have a platform, the better documented it should be to give a full overview of all possibilities and advantages over the competition. However, here we also have a gap —&nbsp;<strong>a weak code documentation</strong>. There is lack of tutorials, examples, and potential use cases with a short explanation. </p>



<p>On the market, there is a&nbsp;<strong>lack of platforms</strong>&nbsp;able to deploy models for mobile applications, where hardware specification plays an important role, since models are running on the device not on the cloud. Many platforms provide a lot of tools for general model deployment, however, only a couple of them are domain-specific platforms e.g for like healthcare or energy sector.</p>



<p>Popular optimization-for-hardware libraries can handle only a small fraction of popular frameworks, which is inconvenient when we deal with big infrastructures and models.</p>



<p>It is also important that switching from one platform to another also plays a big role. We have to always keep in mind&nbsp;<em>how much time it will take to find a new tool</em>&nbsp;vs&nbsp;<em>how much time it will take to transfer the current solution to the new tool</em>&nbsp;vs&nbsp;<em>how much time do we have to the project deadline</em>&nbsp;vs&nbsp;<em>whether the new tool has all features we need</em>.</p>



<p>All those remarks play an important role. However, we have to remember that at the end of the day for the customer the most important thing is the fact that it works. Speed and efficiency are no guarantee of success so it’s important to deliver.</p>



<p></p>



<h2><strong>Problem or a challenge?</strong></h2>



<p>To sum up we might ask,&nbsp;<em>Is the whole AI model deployment process&nbsp;</em><strong><em>a problem</em></strong><em>&nbsp;— a matter of doubt and uncertainty, or&nbsp;</em><strong><em>a challenge</em></strong><em>&nbsp;— an opportunity for success and growth?</em></p>



<p>In my opinion, it is a challenge worth exploring more in detail. My final thoughts are as follows.</p>



<p>Currently there is a small concern who should be responsible for deployments and later maintenance. In most cases ML Engineers / AI developers are not responsible for AI model deployment. Backend engineers are ones who deal with it.</p>



<p>Future platforms have to be transparent in costs and able to version the model. There is a need for the software able to optimize models for specific hardware e.g. mobiles, edge, or websites.</p>



<p>Finally, led by inspirational Leonardo’s da Vinci quote&nbsp;<em>“simplicity is the ultimate sophistication”</em>&nbsp;I need to admit that there is also a place for simple and easy to use tools centered around one particular field of industry or task allowing fast prototyping and testing.</p>


artificial intelligence deployment - a problem or a challenge

Artificial Intelligence model deployment, a problem or a challenge?


<figure class="wp-block-image size-large"><img src="https://data.syndic.ai/wp-content/uploads/2020/06/marin.png" alt="" class="wp-image-36" srcset="https://data.syndicai.co/wp-content/uploads/2020/06/marin.png 1000w, https://data.syndicai.co/wp-content/uploads/2020/06/marin-300x169.png 300w, https://data.syndicai.co/wp-content/uploads/2020/06/marin-768x432.png 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></figure>



<p class="has-text-color has-large-font-size has-light-gray-color">On the mission to simplify the process of making AI models accessible and usable to developers.</p>



<p></p>



<h2 id="9bf8"><strong>Executive summary</strong></h2>



<p>AI model deployment is one of the biggest challenges for Machine Learning engineers nowadays. It&#8217;s a very hard and time-consuming process that requires particular deep tech knowledge in specific areas. In&nbsp;<a href="https://syndic.ai/" target="_blank" rel="noreferrer noopener">syndicai</a>&nbsp;we are on a mission to change that. We are building&nbsp;<strong>a platform that aims to redefine the way developers deploy and implement AI models.</strong>&nbsp;Our goal is to give developers&nbsp;<strong>a simple way from a trained model to actual use-case</strong>&nbsp;with just a couple of clicks. We believe that making AI more accessible and usable will allow engineers to focus on solving big problems more efficiently. Rather than spending time on setting up the infrastructure.</p>



<h2><strong>Problem</strong></h2>



<p>In order to build a machine learning model, we need to collect, clean, and prepare data, as well as train, test, and tune a model. If everything goes according to plan and the model is able to perform the desired task with high accuracy we’re done. Unfortunately&nbsp;<strong>for business people, trained but not deployed model does not bring any value</strong>&nbsp;to a company. Despite the fact that this last step is very important, only a&nbsp;<a href="https://www.forbes.com/sites/cognitiveworld/2020/03/31/modelops-is-the-key-to-enterprise-ai/#7e98863a6f5a" target="_blank" rel="noreferrer noopener">small percentage</a>&nbsp;of AI models reach that final phase. There are&nbsp;<a target="_blank" rel="noreferrer noopener" href="https://medium.com/syndicai/artificial-intelligence-model-deployment-problem-or-challenge-6438ecec20a9">several reasons</a>&nbsp;for that, the main ones are as follows:</p>



<ol><li><strong>Computing<br></strong>Deep learning algorithms require a lot of computing power not only for training but also for model inference. Clever management of those resources is a big challenge and it’s not easy.</li><li><strong>Scalability<br></strong>Some libraries and frameworks used by Data Scientists are not adapted to perform distributed operations. </li><li><strong>Lack of resources to learn<br></strong>Roughly 95% of AI courses on the internet covers only the first part of the Machine Learning workflow finishing with the trained model</li></ol>



<p>There are many more but we present just a few to give a glimpse of the current situation.</p>



<h2><strong>Solution</strong></h2>



<p>We believe in a clever way of putting AI models into production that smoothen the whole process. As true lovers of simplicity and advanced automation we see a huge opportunity in this area. We feel challenged to face that.</p>



<p>Here comes the time to introduce you to our solution to the problem —&nbsp;<a href="https://syndic.ai/" target="_blank" rel="noreferrer noopener">syndicai</a>&nbsp;platform.</p>



<blockquote class="wp-block-quote"><p>Syndicai is on a mission to simplify the process of making AI models accessible and usable to AI developers and engineers. By cutting out the middleman and creating a platform that is so easy to use that even your grandmother can use it.</p></blockquote>



<p>We want every developer (especially ML engineer) to be able to deploy and implement a trained AI model. Without dealing with resource management, scalability, computation, and security.</p>



<h2><strong>Product</strong></h2>



<p>With our platform we took as our main focus in the area of AI model deployment and implementation. Our goal is to make this process smooth and fast.</p>



<blockquote class="wp-block-quote"><p>No middleman, no deep tech knowledge required — you get a simple way from model to actual use-case.</p></blockquote>



<figure class="wp-block-image"><img src="https://miro.medium.com/max/4000/1*4RNekv7TDJvsgYjPPx4_Pw.jpeg" alt="syndicai AI deployment &amp; implementation workflow"/><figcaption>Drag &amp; Drop your AI model to deploy and implement using various frameworks and libraries — syndicai will take care of everything whats in the middle.</figcaption></figure>



<h3><strong>Step 1: Deploy</strong></h3>



<p>In order to deploy a model, first you need to&nbsp;<a href="https://docs.syndic.ai/getting-started/quick-start" target="_blank" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener">place two additional files&nbsp;</a>in your model directory:&nbsp;<strong>syndicai.py</strong>&nbsp;which contains only python function and&nbsp;<strong>requirements.txt</strong>. Those files are necessary to recreate the working environment and allow to access the model by web service. When it’s done you just&nbsp;<strong>drag &amp; drop your trained model on the platform.&nbsp;</strong>The whole process takes roughly 20min.</p>



<h3><strong>Step 2: Implement</strong></h3>



<p>Developers who want to implement a model get specially generated code snippet ready-made to copy and paste in different environments.</p>



<h2><strong>Closing remarks</strong></h2>



<p>Our vision is to change the way developers implement AI models into production by allowing them to easily share their work and focus on their proficiency instead of building infrastructure.</p>



<p>Currently, we are at the stage of testing our alpha version of the platform, so if you feel inspired and curious about the upcoming stages or would you like to test the platform before launching — feel free to join us on&nbsp;<a href="https://join.slack.com/t/syndicai/shared_invite/zt-eqbfjmyo-BCNw0gDryzU1A_6GH7xyYw" target="_blank" rel="noreferrer noopener">slack</a>&nbsp;or leave us&nbsp;<a href="https://syndic.ai/" target="_blank" rel="noreferrer noopener">an email</a>.</p>


Blog

The latest news about AI in production from the Syndicai team.

How to deploy a PyTorch model in minutes

What is Machine Learning Monitoring in Production?

What exactly is Machine Learning Operations?

Deep Learning algorithms worth to follow in 2020

What I learned from the AI Nextgrid Seed Program?

Artificial Intelligence model deployment, a problem or a challenge?

Syndicai — the future of AI deployment

Deploy & Integrate AI

Focus on AI building and delivery, not an infrastructure setup.