Data engineering is the AI engineer most vendors don't hire
The dominant share of the work that decides whether a mid-market AI system reaches production is data plumbing — not modelling. Here is what that work actually looks like.
If you are about to sign a vendor for a custom AI build inside a company with a real ERP, a real warehouse, real legacy data layers, and real users — and the vendor's strongest pitch is the seniority of their ML team — you are looking at a project that will struggle to reach production. The interesting engineering work in a mid-market AI project is almost never the model. It is the data pipeline, the failure modes, and the integration with the system of record.
This article describes the work that actually decides whether the system ships. Not modelling. Plumbing. Specifically, the four components of data engineering that every mid-market AI production build runs on, and where each one quietly breaks.
Pull from the system of record
The first piece of data engineering in any mid-market AI build is the connection to the customer's system of record. In manufacturing and B2B distribution, this is almost always an ERP — SAP S/4HANA, Microsoft Dynamics 365, or a Polish mid-market ERP layer that an internal IT team has been operating for fifteen years. The data lives in normalised, hand-tuned tables that were designed for accounting, not for AI inputs. The schema is not documented in the way the AI team would like it documented. The fields are named in the customer's domain language, not in English, and the conventions vary by department.
The work, concretely, is to pull the relevant subset of the schema out of the ERP without breaking it, on a schedule the ERP can sustain, and to map it into a shape the downstream AI pipeline can consume. The naïve version uses overnight bulk exports to CSV. The version that survives twelve months uses change-data-capture against a stable read-replica, with explicit field mappings that the customer's internal IT lead has signed off on. The customer's IT lead, in our experience, is the most important alliance the AI vendor builds on the project — they own the integration surface, they understand which fields are reliable, and they will be the person who calls the AI vendor at 7am when the integration starts producing junk.
The most common failure mode at this layer is that the AI vendor never speaks directly to the customer's IT lead. The data extraction is treated as a procurement detail, handled by a junior engineer or a third-party integration partner, and the AI team works from whatever falls out of that pipeline. Six months in, the schema drifts (a field is renamed, a status code is repurposed), the pipeline silently produces wrong data, and the model's outputs degrade in a way that looks like a model problem but is an upstream data problem.
Normalise across catalogues
The second piece is normalising data across sources that were not designed to be compared. In multi-brand distribution, this is the manufacturer catalogues — each manufacturer publishes their products in a different schema, with different product codes, different attribute names, different pricing conventions, and different update cadences. The AI system that recommends a part for a service visit, or that generates a quote for a multi-brand configuration, has to reconcile these into a single addressable catalogue.
The work here is unglamorous and bottomless. It involves: a canonical product-code scheme for the customer (which is invented for the project, because the customer never needed one before), a per-manufacturer mapping table, deduplication logic on partial product codes, a versioning scheme so that a price quoted in March is still recoverable in November, and a change-detection layer that catches when a manufacturer renames a SKU or splits a product into two variants.
In our engagements, this layer is the one where the customer's domain expertise is most directly load-bearing. The model cannot tell the difference between a SKU rename and a new product. The sales rep with twelve years of experience can. The data engineering work is to build the surface that lets the sales rep correct the system, and that lets the system learn from those corrections without overwriting them on the next nightly import.
Detect the changes that matter
The third piece is change-detection on inputs that update without notice. Manufacturer price lists update weekly without versioning. Compliance documents are reissued with the same filename. The customer's own internal data — open quotes, active service jobs, ongoing contracts — changes continuously throughout the working day in ways the AI system needs to respect.
The pattern of work here is to define, per data source, what kind of change matters and what kind does not. A price list whose prices change by less than half a percent is noise. A price list whose prices change by more than five percent is a signal that requires the AI system to invalidate cached quotes that were generated against the old prices. The threshold is not the same across sources, and it cannot be inferred without the customer's domain context.
This is the work where a recent practitioner study by Shankar et al. is unusually concrete. The 2024 study documents how working ML engineers operationalise systems through data preparation, experimentation, evaluation, and continual monitoring, and notes explicitly that the role requires proficiency in both data science and engineering (Shankar et al., 2024). The honest framing is that the boundary between "data engineering" and "ML engineering" in production is administrative. The actual work is the same person doing both, sustainably, with enough domain context to know which change is signal and which is noise.
Build the golden set, then maintain it
The fourth piece — and the one that consulting engagements skip most reliably — is the golden set. A golden set is a frozen collection of representative real inputs with their expected outputs, against which every change to the AI system is evaluated before deployment. For a mid-market quote system, this means a few hundred real quotes that the customer's sales team has produced manually, paired with the structured output the AI system is supposed to produce. For a service-part recommender, it means a few hundred real visits with their actual part requirements.
The work to construct a golden set is roughly two to three weeks of co-located data engineering work with the customer's domain experts, depending on the system. It is not glamorous and it does not run in parallel with model development; it has to happen first or the model development is uncalibrated. Vendors paid for discovery treat the golden set as a delivery-phase artifact and never construct one. Vendors paid to operate the system treat it as a precondition to deployment.
The evaluation harness is what runs the model against the golden set on every change. Without it, the team cannot tell whether a new prompt, a new model version, or a new retrieval strategy improved performance or quietly broke a category of inputs that the engineers were not paying attention to. The Fivetran 2025 enterprise survey reports that nearly half of enterprises blame poor data readiness for delayed or failed AI projects, with more than half saying that the majority of their AI projects fail to deliver (Fivetran, 2025). Informatica's 2025 CDO Insights survey reports that 56% of data leaders name data reliability as a primary barrier to advancing AI pilots (Informatica, 2025). What the numbers describe and what mid-market teams experience are the same thing — the model is not the problem; the input layer is.
The system view
The four components — system-of-record extraction, multi-source normalisation, change detection, and golden-set construction — are the engineering surface that decides whether a mid-market AI system ships into production and stays running. They are not novel. They are not the work that gets featured in AI vendor case studies. They are the work without which no AI vendor case study can be honestly written — and the same operational reality that the on-call question names from the post-deployment side.
The interesting engineering work in an AI project is almost never the model. It is the data pipeline, the failure modes, and the integration with the system of record. The team that takes this seriously enough to staff for it is the team that ships into production. The team that does not is the team whose proposals lead with the seniority of their ML credentials.
Where it breaks
The data engineering layer has three reliable failure modes. The first is schema drift in the source system that the AI pipeline does not detect — a field gets renamed during a quarterly ERP upgrade, the AI vendor was not in the change-control loop, and the system silently produces wrong outputs for two weeks before someone notices. The second is golden-set staleness — the golden set was constructed at deployment and never updated, the model is now being evaluated against a distribution that the production traffic no longer resembles, and every new version passes the evaluation but degrades in real use. The third is change-detection threshold drift — what counted as a meaningful change six months ago no longer counts, the noise floor has risen, and real signals are being filtered out.
All three failures are recoverable. None of them is recoverable by the model team alone. The recovery work is data engineering work, run against the source systems by someone with domain context. If the vendor's contract ends at deployment and nobody owns this work, the failure is permanent — which makes the question of vendor pricing less a procurement detail than a structural prediction of whether this layer will be maintained at all.
What to verify before signing off
Before accepting a mid-market AI system as production-ready, a buyer should be able to verify the following:
- The connection to the system of record is named, documented, and owned — including which fields are pulled, on what cadence, and which member of the customer's internal IT team is the named contact for schema changes.
- The canonical catalogue or canonical data model is documented — and the customer's domain experts have signed off on the field mappings.
- The change-detection thresholds are written down, per data source, with the reasoning for each threshold.
- The golden set exists, contains at least a few hundred representative real cases, was constructed with the customer's domain experts, and is scheduled for quarterly review.
- The evaluation harness runs on every change, blocks deployment if regressions exceed a defined threshold, and reports against the golden set in a format the buyer's team can read.
If any of these five items is missing, the engagement was sold as an AI project and built as a demo. The data engineering layer is what makes the difference visible.
What the credential conversation should actually be about
The credential the buyer should evaluate the vendor on is data engineering depth, not ML credentials. Vendors paid for discovery hire the ML-credentialed; vendors paid to operate the system hire the data engineer. The interesting work is upstream of the model — and the consulting register has been hiding it for ten years because the data work does not look impressive in a strategy deck.
A useful test for any vendor proposal: count the named data integrations, the documented schema decisions, the golden-set construction plan. Then count the references to model architectures, parameter counts, and benchmark scores. The first count, in our experience, predicts whether the system reaches production. The second count predicts whether the proposal photographs well.
The data layer is the layer. The model is the layer the buyer was sold on.