Agents for production lines: Trusted decisions in real time

Agents for production lines: Trusted decisions in real time | Databricks Blog

Skip to main content

Minutes, not hours. When a packaging line trips, every minute counts. A 10-point drop in OEE costs a typical CPG company tens of millions of euros a year. That's not even an hour a shift. Not much room to think; the line manager has to make the recovery call before downstream equipment starves. ProdLine CoPilot reads live state from the Databricks Data Intelligence Platform and gives back an in-shift answer, instead of the next morning.

Unified data core. Zerobus streams OT telemetry into Delta tables; MES, ERP, and LIMS join under Unity Catalog. Domain specialists read live plant state and call real solvers (Monte Carlo, MILP, Bayesian, Pareto) under the same constraints the planners already use.

The team stays in charge. Recommendations come back as draft work orders, holds, and schedule notes. Line manager, quality lead, and maintenance owner approve. Every decision is traceable end to end.

Executive summary

09:14, mid-shift. The filler trips. The line manager has minutes, not hours, before downstream equipment starts to starve. The crew already knows what to do mechanically. The questions that take longer are the planning ones. Can we still hit the shift target? Is it cheaper to push speed afterwards or call overtime? Has the same fault hit this line before, and how did the previous shift recover?

The data to answer all three already exists, scattered across PLCs, SCADA, MES, ERP, and LIMS.

ProdLine CoPilot is built for that window. It reads live state from the Databricks Data Intelligence Platform, routes the question to a domain specialist, and runs the underlying math (schedule recovery, depletion, quality risk). The plan comes back tested against 1,000 scheduling scenarios balancing trade-offs in cost, overtime, and service. The line manager picks. The system drafts the artifacts (work order, hold, schedule note) for approval.

The real problem: data-rich, insight-poor

A typical CPG packaging line (bottling, canning, snacks, cosmetics) has 15–20 machines. When a filler or labeller stops, buffers only cover a few minutes before the line starves and output falls well below nameplate. World-class OEE sits near 85%; many plants are closer to the low-to-mid 70s. At 500 cases/hour on a 24/5 schedule with €10 contribution per case, one OEE point is roughly €300K a year. Close a 10-point gap on one line and you’re in the low millions; across a plant with a dozen lines, that adds up fast.

The data to close that gap already exists:

PLCs and SCADA stream sub-second telemetry.

MES holds every micro-stop, major stop, hold, and changeover record.

ERP carries inventory and schedules.

LIMS owns the quality results.

Those systems do not talk to each other. The people who need answers (line managers, shift leads, planners) often do not write SQL.

The pattern is familiar: shift-end reports first, analyst query the next morning, then an RCA meeting 24 hours after the fact. All while the recovery call (speed, overtime, CIP) was already made during the shift.

Streaming OT to Databricks isn't just a dashboard upgrade. Joining OT with MES, ERP, and LIMS in one governed lakehouse is what lets agents reason over live state, optimize under real constraints, and recommend in-shift instead of after the fact.

Real-time streaming: from morning report to in-shift signal

The old pattern was to stand up Kafka-class plumbing (brokers, partitions, consumer groups) just to move plant data. Zerobus Ingest replaces that. It's push-based and serverless. Anything that can issue gRPC or REST calls (PLC gateway, historian connector, edge box) lands rows in Unity Catalog Delta tables.

No brokers, no partitions; you scale by opening more connections. Pair it with Lakeflow Spark Declarative Pipelines and the standard Bronze, Silver, Gold medallion layout for telemetry, quality signals, events, and inventory.

MES, ERP, and LIMS land on a slower cadence than sub-second OT (mirror, batch, or CDC), but they sit alongside the OT tables under one governed catalog rather than in a separate warehouse.

Once data is in, the same tables feed SQL, Genie, AI Search, Model Serving, and agents, with shared lineage under Unity Catalog. Predictive signals, schedule recovery, and downstream analytics all read from these governed tables, so each new capability writes against the existing copy instead of provisioning its own.

Zerobus + Delta handles near-real-time ingest at single-digit-second latency, governed under Unity Catalog. For the live UI in this demo, the producer also writes directly to Lakebase: a shortcut to get the real-time feel today, not the long-term pattern. The millisecond read side on those same Delta tables is what Lakehouse//RT, Databricks' real-time warehouse on the lakehouse, will handle.

Why Databricks

Many plants run reporting in one place and models in another, so the line itself ends up with more than one version of the truth across systems. That split breaks shift-time copilots:

Latency builds at every handoff between systems.

Teams pay a reconciliation tax to align extracts.

Permissions drift between data sources.

Lakehouse on Databricks closes that split at each of those three points. There's one copy of the data (open Delta on cloud storage, not a separate extract per workload). Governance lives on that copy in Unity Catalog, so the analyst's permissions and the agent's permissions come from the same source. And streaming, SQL, AI, and serving all run on one foundation, so the morning report and the live screen show the same number.

Specialists and optimizers read the same governed tables your pipelines maintain. There is no separate AI database.

Anatomy of a production line agent system

The orchestrator

The orchestrator is the front door. It accepts a natural-language question, loads current state from Unity Catalog (machines, events, schedule, inventory, quality, constraints), and routes intent to the right specialist.

Every call reads the latest UC state before the LLM starts. The system recommends and drafts artifacts (tickets, approvals, shift notes). Execution stays with line manager, quality, and maintenance.

Short conversation memory lives in Lakebase, the past-incident corpus in AI Search. Model Serving serves the models and MLflow traces every call.

Why a roster, not one big agent

One generic agent over-simplifies or loses focus. Downtime RCA, inventory, and schedule math need different data and different math. A roster of specialists keeps each prompt narrow and each tool targeted to the question.

For example, Downtime Analyst does not burn context on inventory tables, and Schedule Optimizer does not pull raw quality check rows the way Quality does.

SpecialistWhat it does

Downtime AnalystRoot cause, cascade across machines, recovery priority; events + sensors

Quality SpecialistSPC on fill, torque, labels, case weight; hold/release; Bayesian risk

Supply Chain AdvisorTens of line inputs (e.g. labels, film, closures, adhesives, process chemicals) — burn rates, depletion, reorder urgency

OEE CoachAvailability / performance / quality loss; Pareto; Genie for trends

Schedule OptimizerMILP / stochastic recovery plans; trade-offs: cost, schedule/service risk, throughput

Maintenance PredictorAnomalies (Z-score, IQR); RUL-style signals; PM trade-offs

Strategic AdvisorMulti-shift trends; improvement roadmap; capex/opex framing; benchmarking

Shift BriefingPre-shift huddle / post-shift handover readouts; Genie-friendly, mobile-first summarisation

Tools the specialists actually call

Specialists call a small fixed set of tools. SQL Query and Genie Space pull governed reads, the same way the rest of the org does. The Calculator runs OEE, recovery, and depletion math in Python (NumPy and Pandas) over telemetry from Databricks SQL. An Anomaly Detector runs Z-score and interquartile range (IQR) over rolling windows directly on those tables. Plan & Constraints holds the per-line speed limits, CIP windows, changeover rules, and overtime policy. Similar Cases retrieves historical incidents from Databricks AI Search.

Routing to a real solver, not just chat

Many manufacturing copilots are thin LLM wrappers. ProdLine routes to real solvers (the kind operations research teams use), via natural language.

OptimizerMethodWhat it solves

Schedule RecoveryMILP (OR-Tools SCIP)Speed, OT, CIP — optimal under stated model, not a vague heuristic

Stochastic ScheduleSAA + scenariosRobust plan across OEE / micro-stop variability

Production ForecastMonte Carlo (e.g. 1,000 paths)P10/P50/P90 completion bands from history

Quality RiskBayesian CPTRisk score + drivers

OEE Loss AnalysisParetoRank losses by magnitude / ROI

Multi-Shift PlannerSequential optimizationCross-shift speed, OT, CIP, PM

RUL EstimatorTrend extrapolationPM timing trade-off

Human-in-the-loop: close the loop without extra work

Nothing executes without a human signing off. The line manager owns the recovery, quality owns hold and release, and maintenance owns the work order. The point is to cut the cognitive load of bouncing between spreadsheets, radios, and dashboards, not to remove the production manager.

It has to answer three questions in under a minute: what is happening, what are the realistic options, and what does each option cost in throughput, overtime, quality, and service.

Approval gates (by design):

RoleApproves

Line managerRecovery: speed, overtime, schedule

QualityHold/release, deviations

MaintenanceWork scope and timing

The current demo covers the reasoning and recommendation loop. The next step is to close the loop with system write-backs, all designed as drafts rather than auto-control.

The CMMS handoff is a draft work order (diagnosed fault, recommended scope, target time, required parts) for the planner to schedule. For quality, the QMS and LIMS receive a pre-filled deviation record (lot, machine, sample IDs, severity, recommended disposition) that the quality lead reviews and dispositions. MES and the advanced planning and scheduling (APS) system pick up a draft schedule update with speed adjustments, overtime, sequence changes, and the recovery rationale, written back for shift execution.

Traceability follows the same roadmap: each recommendation stores its inputs, assumptions, constraints, approver, and outcome end-to-end, supporting handover between shifts and continuous improvement.

From pilot line to multi-plant

The hardest part of multi-plant rollout is the data, not the AI. Every plant has its own machines, SOPs, and LIMS schema. What makes the second plant additive rather than a parallel project is the streaming layer underneath: each plant lands on the same Zerobus pattern, medallion layout, and Unity Catalog governance, with its own tables and Genie Space under a dedicated namespace.

The optimizers stay parameterized. A line_constraints table drives speed bounds, overtime limits, CIP windows, and changeovers, so changing the data changes the behavior, no redeploy required.

The same foundation funds the next use case. Energy and sustainability read the same telemetry, supplier quality builds on the LIMS join, and safety builds on the event stream. Each new project lands on infrastructure the first one already paid for, instead of standing up a parallel platform.

Call to Action

Clone the code repository and run databricks bundle deploy in your own workspace. To use your own plant data, point the producer at your historian instead of the simulator; Zerobus, the medallion layout, agent tools, and human-in-the-loop drafts stay put. Open prodline_copilot_film.html in a browser for a two-minutes animated walkthrough before you clone.

Are you interested in building your own line monitoring assistant and would like to find out more? Contact your Databricks account representative. A Databricks specialist can also help scope bringing OT, MES, ERP, and LIMS into one governed lakehouse.

Glossary

Acronyms used in this post, alphabetical.

APS – Advanced Planning & Scheduling

CDC – Change Data Capture

CIP – Clean-In-Place (cleaning cycle on a production line)

CMMS – Computerized Maintenance Management System

CPG – Consumer Packaged Goods

ERP – Enterprise Resource Planning

IQR – Interquartile Range

LIMS – Laboratory Information Management System

LLM – Large Language Model

MES – Manufacturing Execution System

MILP – Mixed-Integer Linear Programming

OEE – Overall Equipment Effectiveness

OT – Operational Technology (the plant-floor systems side of IT/OT).

PLC – Programmable Logic Controller

QMS – Quality Management System

RCA – Root Cause Analysis

SCADA – Supervisory Control And Data Acquisition

SOP – Standard Operating Procedure

Related Databricks Blogs

Developing Near Real-Time Apps: Zerobus and Lakebase

Building Managed Enterprise AI Frameworks

Maximizing Manufacturing's ROI

How to Build Digital Twins for Operational Efficiency

Jumpstart your Data Modeling with Databricks Industry Data Models

Get the latest posts in your inbox

Subscribe to our blog and get the latest posts delivered to your inbox.

Sign up

View all blogs