A robot arm that can fold a shirt did not get that skill from a rulebook. Nobody wrote a function called fold_sleeve(). Instead, somewhere upstream, a human sat at a control rig and folded hundreds of shirts through the robot's own joints, or a simulator ran millions of failed attempts overnight, or both. The skill emerged from data and repetition, not from an engineer enumerating every case in advance. That shift — from hand-coded motion to learned behavior — is the single biggest change in robotics over the past decade, and it explains why the field suddenly looks less like industrial automation and more like machine learning with a body attached.
This post walks through how that learning actually happens: the mechanics of teleoperation, how imitation learning turns human demonstrations into a trained policy, where reinforcement learning and simulation fit in, and where the whole approach still breaks down.
Why robots couldn't just be programmed
For decades, industrial robots were programmed the traditional way: an engineer specified exact coordinates, joint angles, and timing, and the robot replayed that script with high precision. This works extremely well in a controlled setting — a car door always arrives at the same position on the line, a weld point never moves. It fails almost immediately outside that setting. A warehouse robot that has to pick an object it has never seen, sitting at an angle nobody scripted for, on a surface with unpredictable friction, cannot be handled with a fixed motion program. The number of cases is unbounded.
Two things had to change to make general-purpose manipulation possible:
- Perception got good enough to generalize. Vision models trained on large image datasets can recognize and localize novel objects well enough to inform a grasp, even without a human pre-specifying what that object is.
- Control got treated as a learning problem. Instead of writing the motion, engineers started training a model — usually a neural network — to output the motion, conditioned on what the robot currently sees and where its joints are.
That second shift is the "learning" in how robots learn. The rest of this post is about the different ways engineers generate the data that trains that model, because the data source is what most distinguishes one approach from another.
Teleoperation: the human as the first model
Teleoperation means a human directly drives the robot's body in real time, usually through a matching set of controllers, a VR headset, an exoskeleton rig, or even a second small robotic arm that mirrors the operator's hand movements. The operator sees what the robot's cameras see and moves as if their own hand were the gripper.
Teleoperation serves two very different purposes depending on context:
- Direct operation. In hazardous or remote environments — bomb disposal, undersea inspection, surgical robotics — a human stays in the loop for every action, and the robot never has to act autonomously. This is teleoperation as a final product, not a training step.
- Data collection for autonomy. Increasingly, teleoperation is used purely to generate training examples. An operator performs a task — say, placing a part in a fixture — dozens or hundreds of times, and every recorded trajectory (joint positions, camera frames, gripper state) becomes a labeled example of "correct" behavior. The robot is not meant to stay teleoperated; the recordings are the point.
The second use case is what has exploded recently, because it turns an expensive, scarce resource — real robot experience — into something that scales roughly with how many people you can put behind control rigs. Some robotics labs have built entire fleets of teleoperation stations specifically to mass-produce demonstration data, treating it more like a data-labeling pipeline than a robotics R&D exercise.
The catch is cost. Teleoperation requires the physical robot, a skilled or at least trained operator, and real time — you cannot speed up a human folding a shirt the way you can speed up a simulation. It produces the highest-quality, most realistic data available, but it is the slowest and most expensive way to get it.
Imitation learning: turning demonstrations into a policy
Once you have a pile of demonstrations — whether from teleoperation, motion capture, or even video of humans performing a task — imitation learning is the process of training a model to reproduce that behavior on its own, without a human in the loop at execution time.
The most common form is behavior cloning: treat each recorded (observation, action) pair as a training example, and train a neural network via supervised learning to predict the action given the observation. If the network sees a similar frame to one in training, it should output a similar action. Modern versions of this use the same transformer-style architectures behind large language models, sometimes literally called "vision-language-action" models, because they take in camera images and even natural-language instructions and output motor commands as a single learned function.
Imitation learning has a well-known weakness: compounding error. A cloned policy is only trained on the states a human demonstrator actually visited. If the robot drifts slightly off that path — a grasp that's a centimeter off, a joint angle the demonstrator never hit — it enters a state the model was never trained on, and its predictions get worse, which pushes it further off-distribution, which makes predictions worse still. Small mistakes snowball. This is why raw behavior cloning historically struggled on long or precise tasks, even with lots of demonstration data.
Two families of fixes have made imitation learning much more reliable:
- Better data collection strategies, such as having the model act while a human corrects it in real time (interactive imitation learning), so the training data includes recovery behavior, not just perfect execution.
- More expressive action models, such as diffusion-based policies that predict a distribution over plausible next actions rather than a single deterministic one, which better captures the genuine variability in how a task can be done and tends to degrade more gracefully near the edge of the training distribution.
Reinforcement learning and the role of simulation
Imitation learning teaches a robot to copy what it saw. Reinforcement learning (RL) teaches a robot to optimize for an outcome, by letting it try actions, observe a reward signal, and adjust its behavior to get more reward over time. RL does not need a human demonstrator at all — in principle, a robot can discover a grasping strategy nobody ever showed it, simply by being rewarded when the object ends up in the bin.
The obvious problem is that RL typically needs an enormous number of trial-and-error attempts to converge, and trial-and-error on a physical robot is slow, wears out hardware, and can be unsafe — you do not want a robot arm "exploring" near a person or an expensive fixture. This is why almost all serious RL for robotics happens primarily in simulation, where a physics engine stands in for the real world and can run thousands of parallel attempts far faster than real time, with zero risk to hardware.
Training entirely in simulation creates a new problem, known as the reality gap or sim-to-real gap: a policy that's excellent in simulation often performs worse, or fails outright, on the real robot, because the simulator's physics, textures, sensor noise, and object properties never perfectly match reality. Friction coefficients, cable dynamics, and camera artifacts are notoriously hard to simulate exactly.
The standard mitigation is domain randomization: instead of training in one carefully tuned simulated environment, train across thousands of randomized variants — different lighting, different friction values, different object masses, slightly wrong camera calibration — so the resulting policy is forced to be robust to variation rather than overfit to one simulator's specific quirks. A policy that works across a wide spread of simulated conditions tends to transfer to the real world reasonably well, because reality just looks like "one more variant" it was already trained to handle.
In practice, most modern robot learning pipelines combine all three ingredients rather than picking one:
| Approach | Data source | Strength | Main weakness |
|---|---|---|---|
| Teleoperation | Human directly operates the robot | Highest-fidelity, realistic data | Slow and expensive to collect at scale |
| Imitation learning | Human or teleoperated demonstrations | Learns quickly from relatively little data | Compounding error off the demonstrated path |
| Reinforcement learning | Trial-and-error against a reward signal | Can discover strategies no human demonstrated | Needs huge numbers of attempts; unsafe on real hardware |
| Sim-to-real (RL + domain randomization) | Simulated trial-and-error, randomized | Cheap, fast, safe to run at scale | Reality gap; simulator never matches the real world exactly |
A typical modern pipeline: pretrain a policy on a broad base of demonstrations (imitation learning) to give it reasonable general behavior, then fine-tune or refine it with reinforcement learning — either in simulation, on the real robot in a constrained setting, or both — to sharpen performance on the specific task and correct the weaknesses imitation alone couldn't fix.
Why this matters for businesses and builders right now
For a long time, deploying a robot for a new task meant months of engineering: custom fixtures, hand-tuned motion scripts, and re-engineering every time the task changed even slightly. Learning-based approaches change the cost structure of that work. If a robot's skill comes from a trained policy rather than a hand-written program, adapting it to a new object or a new variant of a task can, in principle, mean collecting more demonstrations or running more simulated training rather than rewriting code from scratch.
This has practical consequences for teams evaluating robotics investments:
- The bottleneck moves from engineering to data. The hard part of a learning-based robotics project is increasingly not "can we control the arm" but "can we collect or generate enough good demonstration and interaction data for the model to generalize." That changes hiring needs, project timelines, and where budget should go.
- Generalization is now a real, if imperfect, selling point. A policy trained across many objects and scenes can often handle a genuinely new object reasonably well, something a hand-coded pick-and-place script never could. This matters for warehouses, kitchens, and any environment with high variability, though "reasonably well" still falls short of human reliability in most deployed systems today.
- Simulation infrastructure is now a competitive asset, not a research nicety. Companies investing in high-fidelity, fast simulators with domain randomization are effectively building a data factory that produces far more training experience per dollar than physical robot time ever could.
- Safety and evaluation get harder, not easier. A hand-coded robot fails in predictable, debuggable ways. A learned policy can fail in ways that are hard to anticipate or explain, which raises the bar for testing before deployment near people or expensive equipment.
None of this means teleoperation and manual engineering are going away. Most deployed systems today still lean on a mix: learned perception and grasping layered on top of more traditional, hand-engineered motion planning and safety layers, with humans able to take over via teleoperation when the autonomous policy is uncertain or the task is too novel to trust to the model.
Limitations and open questions
It's worth being direct about where this approach still falls short, because the marketing around robot learning often outruns the reality.
- Data is still scarce relative to what these models need. Language and image models train on internet-scale datasets. There is no equivalent internet-scale corpus of robot interaction data — every trajectory has to be physically generated or simulated, which is orders of magnitude more expensive per example than scraping text or images.
- Cross-embodiment transfer is unsolved. A policy trained on one robot arm, with its particular set of joints, gripper, and camera placement, does not automatically work on a differently shaped robot. Researchers are actively working on models that generalize across robot bodies, but this is still an open problem rather than a solved one.
- Long-horizon, multi-step tasks remain hard. Compounding error and reward sparsity both get worse as tasks get longer. A robot that reliably picks up a single object may still struggle with a ten-step assembly sequence where one early mistake derails everything after it.
- Evaluation is genuinely difficult. Because learned policies are statistical rather than rule-based, there's no simple way to prove a policy is safe or correct across the full space of situations it might encounter, unlike a hand-written script that can at least be read and audited line by line.
- The reality gap has not fully closed. Domain randomization and better simulators have narrowed it substantially, but sim-trained policies still commonly need real-world fine-tuning before they're reliable, especially for contact-rich tasks like assembly or manipulation of deformable objects like fabric or cable.
What to watch next
A few threads are worth tracking if you want to understand where robot learning is heading:
- Foundation models for robotics. Just as large language models generalize across many text tasks from one pretrained base, researchers are building "generalist" robot policies pretrained on data pooled across many tasks, robots, and even datasets scraped from human video, with the hope that a single base model can be fine-tuned quickly for new tasks rather than trained from scratch each time.
- Learning from video, not just teleoperation. Because teleoperated data is so expensive, there's active work on extracting useful action information from ordinary human video — cooking videos, assembly tutorials — where a robot never touched anything, to supplement or partially replace teleoperated demonstrations.
- Better simulators and digital twins. As physics engines and rendering get more accurate and cheaper to run at scale, the reality gap should keep narrowing, making simulation an even larger share of total training experience relative to physical robot time.
- Standardized evaluation. As learned policies move from research demos to deployed systems, expect more pressure for shared benchmarks and safety-testing standards specific to learned robot behavior, closer to how software testing and certification work in other safety-relevant fields.
FAQ
What is the difference between teleoperation and imitation learning?
Teleoperation is a human directly controlling a robot in real time, whether as the final mode of operation or purely to record demonstrations. Imitation learning is the separate step of training a model on those recorded demonstrations so the robot can perform the task on its own, without a human driving it during execution.
Can robots learn without any human demonstrations at all?
Yes, through reinforcement learning, where a robot improves by trial and error against a reward signal rather than copying a human. In practice most systems combine both: imitation learning for a reasonable starting policy, and reinforcement learning to refine it further.
Why do robots train mostly in simulation instead of the real world?
Simulation lets a robot attempt a task thousands of times in parallel, far faster than real time, without wearing out hardware or risking damage. The tradeoff is the "reality gap" — simulated physics never perfectly match the real world, so simulated training alone usually isn't enough for reliable real-world performance.
What is domain randomization and why does it help?
Domain randomization means training a policy across many randomized versions of a simulated environment — varying lighting, friction, object properties, and camera calibration — instead of one fixed setup. It forces the policy to become robust to variation, which tends to make it transfer better to the real world instead of overfitting to one simulator's exact conditions.
Why does a robot's performance suddenly get worse when it makes a small mistake?
This is compounding error, a known weakness of imitation learning. A cloned policy only learned from the exact states a human demonstrator visited, so a small deviation pushes it into unfamiliar territory, where its predictions get less reliable and the error tends to grow rather than self-correct.
Are learned robot policies safe to deploy around people?
It depends heavily on the task and how the system was tested; learned policies fail in less predictable ways than hand-coded scripts, which makes safety validation harder. Most current deployments pair learned perception and manipulation with traditional, auditable safety layers and human oversight rather than trusting a learned policy end to end.
Will robots eventually learn everything from watching video, without physical practice?
Video is a promising supplement, especially for cutting the cost of data collection, but it lacks the precise force, contact, and proprioceptive information a robot gets from actually performing a task. Most researchers expect video-derived learning to reduce, not eliminate, the amount of physical or simulated practice a robot needs.
Teams evaluating whether a learning-based approach makes sense for a specific manipulation or automation problem can get hands-on help scoping it from Woyce Technologies.