On-Robot Fine-Tuning After Factory Deployment
Robots need real-world fine-tuning after deployment to handle novel conditions.
Transfer, not raw capability, separates a convincing robot demo from a robot that survives six months on a factory floor. Most demos show a robot fine-tuned on one task in one setting, and per Epoch AI's 2026 capability report, transfer to new tasks or new environments should never be assumed unless someone has actually shown it. A robot trained on task-specific data with limited model capacity tends to fall apart the moment conditions shift, a point a February 2024 arXiv survey on foundation models in robotics (led by researchers at Midea Group) makes plainly about generalizability in dynamic real-world scenarios.
For years, the field treated shipping the robot as the finish line. That was always backwards. The real difficulty starts once the machine is on-site, facing object geometries nobody photographed during data collection, lighting that shifts by the hour, surfaces that weren't in any training set, and human coworkers whose habits change week to week. Cross-embodiment transfer makes this worse: a policy trained on one robot arm carries over imperfectly to a different arm, even when the task is identical on paper. Epoch AI's report notes that robots also run slower than humans at most tasks, but speed isn't the binding constraint here. Reliability and adaptability decide whether a deployment survives, and the rest of this piece works through what has to happen after the robot arrives, since no amount of training beforehand can fully prepare it for the place it's about to work.
What broad pretraining actually gives a robot, and what it leaves unfinished
Foundation models are now the default starting point for robot manipulation. Every company working on the hardest problems, household tasks and complex industrial tasks alike, builds on a pretrained foundation model rather than training a policy from zero, according to Epoch AI's 2026 report. The bet is simple: a model exposed to broad, varied data during pretraining should land in a better starting position than a model that only ever saw one task in one room.
The evidence backs the bet, at least directionally. Toyota Research Institute found that a fine-tuned pretrained model matched or beat single-task specialists across 1,800 real-world rollouts and 47,000 simulation rollouts, coming out ahead or even on nearly every task tested (Epoch AI, 2026). Pretraining hands a robot generalized sensorimotor priors: object recognition, a working sense of spatial layout, the ability to follow an instruction phrased in plain language. It does not hand the robot the exact grip force needed for a specific conveyor belt, the lighting profile of a particular warehouse bay at 6 a.m., or the precise sequence a customer's assembly line expects. Treating pretraining as the whole solution is the single most common mistake teams make right now, and it's the one that shows up fastest once the robot hits a real floor: the gap shows up the moment the robot encounters conditions that weren't present during data collection.
Epoch AI draws a direct comparison to what happened in natural language processing: at some point, fine-tuning a pretrained language model started beating models trained from scratch on the target task, and that crossover reshaped how the field worked. Early evidence in the 2026 report suggests the same crossover is now arriving for robot manipulation. Foundation models shrink the distance between "knows nothing" and "works here." They don't close it. That gap is structural, not a bug waiting on a patch, which is exactly why post-deployment adaptation belongs in the system from day one instead of getting bolted on later as a fix.
The progression from single-task robots to models that can be adapted after deployment
Two things have been improving at the same time, and the overlap between them is what makes this approach workable now. One axis is capability: single-task policies gave way to multi-task vision-language-action models, which gave way to cross-embodiment generalist policies, which are now feeding into whole-body humanoid control. The other axis is accessibility: models that once demanded data-center-scale compute are increasingly available open-source and fine-tunable on a single consumer GPU.
Neither axis alone would justify on-robot fine-tuning as a serious strategy. Together, they do. Four years ago the compute requirement alone would have ruled the whole idea out for most integrators, full stop.
Vision-language-action models, VLAs for short, are the dominant architecture right now. A single network takes in visual input and a natural-language instruction and outputs motor commands directly, with no separate perception and planning stack bolted on afterward. The scale here isn't small: millions of robots are already working in factories worldwide, with growth projected to continue. At that scale, per-deployment adaptation stops being a research curiosity and turns into an economic question. FANUC and ABB together account for roughly a quarter of global industrial robot production, and their move to integrate NVIDIA's Isaac AI platform, announced at GTC 2026, signals that the largest players in industrial robotics have committed to AI-driven adaptation as standard practice, not an experiment.
How fine-tuning actually works once a robot is in the field
The practical playbook that's emerged by 2026 is hybrid: pretrain in simulation to build basic sensorimotor competence, then fine-tune on a small set of real demonstrations gathered in the actual deployment environment. Even a strong foundation model gets noticeably better from watching the real robot handle the real task on the real floor, not a simulated stand-in.
The fine-tuning step splits into a few approaches, and they trade off against each other in predictable ways. Full fine-tuning updates every weight in the model. It has the highest ceiling on performance, but it demands real compute and storage, and it carries a specific risk called catastrophic forgetting: the model gets better at the new task while quietly losing ground on tasks it used to handle fine. Parameter-efficient fine-tuning, LoRA being the most common variant, freezes almost all of the model and updates only a small subset of parameters. That cuts memory and compute needs sharply, and per Nebius, LoRA can shrink the number of trainable parameters by orders of magnitude, which is exactly what makes fine-tuning on edge hardware possible at all. Sequential fine-tuning takes a slower, layered approach, adapting the model progressively to more specialized tasks while trying to hold onto what it already knows, a good fit for a deployment site whose demands keep shifting.
Full fine-tuning is the wrong call for almost any robot already in the field, and that needs saying plainly rather than hedged: the compute isn't there on a Jetson-class edge box, and the forgetting risk isn't worth it when a smaller update does the job just as well. Given the hardware actually sitting on a deployed robot, LoRA and similar PEFT methods have become the practical default, not a nice-to-have. The bigger structural advantage across all of this is data efficiency: fine-tuning needs a much smaller dataset than training from scratch, which means the demonstrations can realistically get collected on-site instead of shipped off to a data center. That data comes from teleoperation, from a human physically guiding the robot's arm through kinesthetic demonstration, or from imitation learning pipelines, none of which require a robotics PhD standing at the controls. The Octo model, built by the Berkeley Robot Learning Lab and sized at 93 million parameters, makes the point concretely: its small footprint lets it fine-tune in about 30 minutes.
Running fine-tuned models on the robot itself: why edge inference matters
Sending every inference call to the cloud is a bad idea for most deployed robots, and not for one reason but several at once: latency, the reliability of a factory floor's network connection, data privacy concerns around video and sensor feeds, and the plain cost of running that traffic continuously. Keeping inference on-device closes the control loop locally, which matters most for reactive motor control that has to respond faster than a round trip to a cloud server ever could.
Edge inference deployments show what this looks like in practice: inference happens directly on the robot, generating action continuously, with no data-center GPU anywhere in the loop. OpenVLA, a 7-billion-parameter model built through a multi-institution collaboration, has been shown deployable for edge inference on NVIDIA's Jetson AGX Orin. Octo, at 93 million parameters, sits at the small end of the foundation-model spectrum by design, which makes it well suited to that same Jetson AGX Orin hardware for fast inference and quick field fine-tuning.
Google's Gemini Robotics On-Device, released in June 2025, extends the pattern: a multimodal VLA that runs fully offline on the robot itself, taking in visual input and language instructions and producing motor output without a network connection. It's also the first VLA Google DeepMind made available for fine-tuning. None of this would matter if the hardware couldn't keep up, but Jetson-class chips have crossed the point where both inference and fine-tuning are genuinely possible at the edge, not just theoretically possible. NVIDIA's GR00T N1 architecture reflects the same logic in its design: a reasoning backbone paired with a reactive motor controller, because planning and execution run on entirely different clocks.
Platform architectures being built around post-deployment adaptation
NVIDIA's GR00T N1, released in early 2025, is a 2.2-billion-parameter open VLA pretrained on large-scale simulated humanoid episodes inside Isaac Lab and then fine-tuned on real robot data. Commercial licensing arrived later, with N1.7, not at the original March 2025 launch. Partners on the platform include Agility Robotics, Boston Dynamics, 1X Technologies, Mentee Robotics, and NEURA Robotics. GR00T N1.5, announced at COMPUTEX in May 2025, introduced further capability improvements and the GR00T-Dreams blueprint, which cut synthetic data generation time from months down to roughly 36 hours. GR00T N1.7 reached Early Access on April 17, 2026: 3 billion parameters, built on a Cosmos-Reason2-2B backbone paired with a 16-layer diffusion transformer handling low-level motor control, open and commercially licensed.
NVIDIA's Physical AI Data Factory Blueprint takes a different angle on the same problem: an open reference architecture for generating, augmenting, and evaluating training data, aimed squarely at cutting the cost of training and retraining physical AI systems as they scale.
AMD has built an edge-to-cloud pipeline that runs the other direction structurally but solves the same problem: imitation learning data gets collected at the edge on Ryzen GPU-based PCs, fine-tuning happens in the cloud on AMD Instinct MI300X servers running ROCm, and the trained policy ships back down to the edge hardware. That pipeline was demonstrated across food packaging, childcare assistance, allergy aid, and waste classification at Open Robotics Hackathons in Tokyo, Paris, and San Jose during 2025.
XPeng runs a four-workshop pipeline worth naming in full, because it shows a vertically integrated version of the same idea: workshop one handles pretraining and post-training, workshop two distills the model down, workshop three continues pretraining the distilled version, and workshop four deploys the resulting XVLA model to the vehicle. Scale AI's data network, which delivered a substantial volume of physical AI data over the course of 2025 with named partners including Generalist AI and Physical Intelligence, points to something quietly significant: the data pipelines feeding all of this fine-tuning are turning into an industrialized supply chain in their own right, not a one-off project each robot buyer handles alone.
When fine-tuning the model isn't the only path: inference-time adaptation and hybrid control
Not every shift in deployment conditions justifies a full fine-tuning cycle. Treating fine-tuning as the only lever available is a mistake, and a costly one, since it burns compute and engineering time on drift that a lighter intervention would have handled just as well. Sometimes the constraint is time, sometimes it's compute, and sometimes the priority is simply not disturbing a capability the robot already has working correctly.
VLA-Pilot, accepted to IEEE Robotics and Automation Letters in March 2026, offers an alternative: inference-time policy steering that lets a pretrained VLA policy get deployed zero-shot, no fine-tuning step required. It works in three stages. An EPS-CoT module reasons out a task-aligned steering objective from context. An Evolutionary Diffusion process optimizes action proposals. A post-execution reflection step refines behavior iteratively. None of it touches the model's weights.
A separate approach augments classical finite-state machine control with an on-device language model that generates velocity suggestions asynchronously, accepted only when they pass deterministic safety checks, keeping the high-frequency control loop intact with a fast fallback always available. Work presented at the IROS 2025 Workshop on Generative AI for Robotics and Smart Manufacturing on AR-guided robotic taskss and Smart Manufacturing, uses augmented reality for point calibration and a language model to translate spoken commands into machine-executable parameters in real time, letting a human operator steer adaptation without touching a single weight.
Line these up and a spectrum appears: full fine-tuning at one end, zero-shot inference steering at the other, hybrid LLM-augmented control somewhere in between. Full fine-tuning wins when a genuinely new task family shows up and nothing in the existing weights covers it. Inference-time steering wins for the ordinary week-to-week drift in lighting, part geometry, or line speed that doesn't warrant touching the model at all. For most industrial deployments the right answer isn't picking one lever and sticking with it. It's running both, aimed at different problems on the same floor, and knowing which one a given failure calls for.
What the modular view of embodied systems reveals about where fine-tuning fits
A single end-to-end policy model can't carry an entire embodied intelligence system by itself. It needs a surrounding set of functional modules around it, an argument made in a July 2026 arXiv white paper on embodied operators, authored by researchers across JD Technology, Tsinghua, Tianjin, and Beihang. Those operators are reusable, deployable units with clearly defined task semantics and standard input-output interfaces: detection and segmentation, spatial localization and 3D scene understanding, hand motion recovery, task-decision models, planning and control.
Each of those layers can get adapted or swapped out on its own, which means fine-tuning the VLA policy doesn't ripple through a system uniformly. Its effects depend entirely on which operators sit downstream of it. The white paper proposes evaluating the whole system across correctness, end-to-end efficiency, resource use, temporal stability, portability, interface compatibility, deployment reliability, and downstream task utility, a far richer scorecard than accuracy alone. That richer view has a practical payoff: a robot operator paying for a fine-tuning service needs to know exactly which modules changed, what downstream effects followed, and how to confirm the update actually improved the thing it was supposed to improve. The white paper names its open problems plainly too: operator composition, data standardization, VLA safety, edge deployment constraints, real-world application value. On-robot fine-tuning, for all its progress, still doesn't reliably solve everything the label implies, and anyone selling it as a finished capability is overselling it.
What the industrial scale of deployment means for how fine-tuning gets organized
Industrial robot installations are climbing toward a future with millions of AI-capable units in the field, and adapting each one by hand, one at a time, simply doesn't scale. Anyone still planning to fine-tune fleets robot-by-robot is planning around a number that stopped being small years ago. Fleet-level fine-tuning needs centralized data pipelines, version control for whatever policy is currently deployed on which units, and a way to check a fine-tuned model before it spreads out to the rest of the fleet.
XPeng's four-workshop structure offers one answer to that problem, with distillation serving as the hinge step that shrinks a large trained model down to something small enough to actually run at the edge. NVIDIA's Physical AI Data Factory Blueprint attacks the same problem from the infrastructure side, standardizing how data gets generated, augmented, and evaluated across an entire fleet rather than per unit. Scale AI's industrialized collection pipeline points toward the same conclusion from yet another angle: data supply is turning into a managed service, not something each deployment handles from scratch. The benchmark categories from the embodied operators framework, portability, deployment reliability, temporal stability, map almost directly onto fleet management concerns. A fine-tuned policy that works on one unit is only useful if someone can verify it works the same way on the next thousand.
For integrators and platform vendors, the upshot is plain: the value of on-robot fine-tuning can't be separated from the infrastructure that governs, checks, and distributes the updated policy across every unit that needs it. A team that nails the fine-tuning method but skips the version control ends up with a fleet running a dozen silently different policies within a year, which is its own kind of failure.
Where on-robot fine-tuning goes from here
The trajectory across pretraining, edge hardware, platform architecture, and fleet-scale data pipelines points toward one conclusion: deployment was never the finish line, and the industry has largely stopped pretending otherwise. What began as single-task robots retrained from scratch for every new job has become a layered system: broad pretraining for general competence, targeted fine-tuning for the specific floor, inference-time steering for the day-to-day variation that doesn't justify a full retrain.
The open problems named across the embodied operators white paper, operator composition, data standardization, safety at the VLA level, are not small, and nobody serious in this space claims they're close to solved. But the direction is set. A robot that ships today isn't a finished product the way a factory line once assumed it would be. It's a starting point, and everything that happens to it after it lands on the floor is where the actual engineering now lives.
