Build or Buy a platform for MLOps?

In the following article, we will deep dive into the pros and cons of the build or buy MLOps platform dilemma.

Why is this topic so important?

IDC reports that 28% of AI/machine learning projects fail due to a lack of production-ready data and integration of development environments. Many more projects (47%) never go to production and fall before leaving the experimental phase. Therefore, companies more often look for a solution in the form of an MLOps platform that will allow implementing best practices of collaboration and communication between Data Science and DevOps Teams to build, deploy and manage end-to-end ML lifecycles.

However, it is not a simple decision whether to build or buy such a tool. Shall we use an open-source solution or instead go for the ready-made platform? What are the pros & cons of both approaches, and how to use them in practice?

In the following article, we will tackle the following questions, Build or buy MLOps platform, bringing more light to Engineers & Executives involved in AI projects to relate to them when needed.

What exactly do we want to achieve?

Before we deep dive into details, we need to investigate what exactly we are looking for. MLOps is a vast topic, and itself is not something that is already defined. Both in terms of the definition (companies mean slightly different things) and the people who should deal with it (ML Team or Backend Team). The whole topic is still fresh, and the terminology has not yet been defined and established.

In general, when we talk about MLOps, we talk about the ability to move AI models from data scientist's machines or laptops to remote machine clusters at scale. The main goal of that process is to automate the whole workflow taking care of resource management, orchestration, data/model versioning, and monitoring. As well as decrease the friction between Data Science and DevOps to help them collaborate and communicate with each other.

...when we talk about MLOps, we talk about the ability to move AI models from data scientist's machines or laptops to remote machine clusters at scale.

At the minimum, the MLOps platform should offer monitoring, versioning, scaling, and governance. All those components are crucial to perform fluent AI delivery at scale.

What are the pros & cons of both worlds?

We can take two approaches for the MLOps platforms; let's deep dive into both of them.

Build

The main advantage of building an internal platform is that it is tailormade to the company's needs. All necessary features play a crucial role, and you don't have to pay for components that are not in use. In addition, some open-source solutions and extensions allow you to solve the problem highly flexibly on the market. Community members and other development teams highly support all those solutions.

However, the effort to build ML operational infrastructure solutions internally in the company is often underestimated by executives and developer teams who create and maintain software platforms as part of their day-to-day operations. According to the study, the whole process of building a minimum viable product takes, on average, 28 months is based on a scenario in which the personnel prioritizes building an enterprise-ready ML platform on top of handling multiple concurrent projects and tasks outside the scope of the platform.

If you decide to choose this approach, please have a look at the following solutions that are a great alternative in some steps:

• Experiment Tracking: DVC, MLFlow
• Pipelines: Pachyderm, Airflow, Kedro, Luigi, Metaflow
• Serving: Seldon, Cortex, Intel OpenVINO, Nvidia Triton
• Infrastructure: Kubeflow, Kubernetes

Before building a platform, you may want to ask yourself and your team the following questions:

1. Can your team commit full-time to work on platform development, maintenance, and tech support?
2. Can you effort delays in providing a real business value that is at the core of your business?
3. Did you plan in detail what is necessary to build to provide real value for your business?

Buy

In contrast, building a platform, a bought one, requires fewer upfront resources and reduces the maintenance and burden of content updates. Therefore, costs and time are drastically reduced. In addition, you also get a proven enterprise-ready solution instead of a minimum viable build solution that is secure and tested. Model, data, and code governance is getting much more important, and not everybody is paying attention to this. With the paid solution, you have that confidence and security not to have to worry about it.

However, even though there are many pros, the obvious number one cons is a price that very often needs to be covered up front. For many companies, this is the reason why they decide to go for the build solution. Nevertheless, it is also important to mention that you lose a little bit of control over it going for a paid solution, and you can’t change freely change components and configure the platform as you wish. This is, of course, not the case for smaller companies that can adjust a lot of parts and features according to the client’s needs while they work closely with the customer.

Before building a platform, you may want to ask yourself and your team the following questions:

1. Do we need to pay for all those presented features, or are we just looking for one particular thing?
2. What do we value most (Ease-of-use, low cost, high flexibility, popularity...)?
3. How will we integrate that solution with our ongoing projects and tools? Do we need to buy any additional connectors?

How to use them in practice?

Talking to companies, we can distinguish three takeoffs regarding the build or buy MLOps platform dilemma.

Experiments (build) + Operations (build)

The first approach is that companies build everything in-house out of open-source components to control the whole AI workflow fully. Then, the solution is fully custom and responds to the company's needs.

• Example 1: MLFlow + Kubeflow
• Example 2: DVC + Seldon
• Example 3: DVC + Intel OpenVINO + Luigi + Kubernetes

Experiments (build) + Operations (buy)

The second approach is the most common and most often implemented by companies. Since the Experimental phase is pretty, well-established companies decided to implement only the Operational part that allows them to automate the workflow, apply CI/CD and orchestrate the infrastructure without taking care of maintenance.

• Example 1: MLFlow + Syndicai
• Example 2: ClearML + Syndicai
• Example 3: MLFlow + Cloud Run

Experiments (buy) + Operations (buy)

The last approach aims to ultimately refine the style and flow of the work currently in the company bringing best practices on the whole ML lifecycle. The experimental phase is tiny connected with the operational one so that everything happens in one place. Data Scientists work closely with DevOps teams using the same tools.

• Example 1: Amazon SageMaker
• Example 2: Weights & Biases
• Example 3: Cnvrg.io

What should I do?

Finally, should we build or buy MLOps platform? There is no right or wrong solution. However, it is essential to know all drawbacks and possible pitfalls that AI teams might encounter down the road.

No matter the solution, the ultimate goal is to get the shortest time-to-value for ML while reducing the risk of failure or stalled efforts. Building an internal platform will be the longest route by far and may not deliver on value add. However, a flexible model management platform designed by an experienced team and built for evolving complexity vastly reduces risk. It ensures your company can extract the most value from your machine learning models from day one.

If you want to learn more about both approaches:

• please schedule a Free Consultation with our Specialists
• takes a look at our talk from Data Science Summit ML Edition.