• one calibration episode in, verified robot data out •

Agentic Data Farm:
GPT-6 Astra as a Cost-Efficient Teleoperator

Kewei Zhang1*, Juncheng Ma1*, Guodong Lv1, Hongyu Zhang1, Yichen Liu1, Jiaqi Hou4, Ye Huang1, Chuanrui Zhang2, Yuxuan Chen1, Ruiyang Si1, Wenbo Huang1, Shukai Gong1, Feixiang Ruan1, Yintianrun Zhang1, Xuanran Zhai3, Yufan Deng1, Enze Xie2, Duomin Wang2, Daquan Zhou1†
1Peking University 2NVIDIA 3National University of Singapore 4Tsinghua University
*Equal contribution   †Corresponding author
Peking University NVIDIA National University of Singapore Tsinghua University
Paper arXiv Code Data
Real robot, unseen scene
87 vs 46 / 100
successes of π0.5 policies trained only on farm data, against the same number of teleoperated episodes, under a tablecloth and coloured light absent from both training sets.
The results →
No person in the loop
$45 per task
mean one-time agent cost to adapt a new task, from one calibration episode to 100 qualified simulated episodes and a render contract, without human intervention.
The farm →
Cheaper than teleoperation
$1.74 per episode
for 100 episodes of a task, the farm is cheaper whenever a teleoperated episode costs more than this (mean over five tasks; $0.71–2.89), in the original scene or a new one.
The cost →
What the farm delivers: simulator vs. rendered frames for five tasks, scene variation by text, real-robot successes, and cost.
What the farm delivers. (a) The same frame of a simulator rollout and of our rendering for each of the five tasks; the rendering is photographic and keeps the simulated arm pose, objects, and contacts (insets enlarge the dashed regions). (b) From a request and one calibration episode, the agent builds the task in simulation, and the video model renders each rollout in the original scene or in a new scene described by text. (c) Real-robot successes of policies trained only on farm data or on the same number of teleoperated episodes, in the original scene and under unseen appearance. (d) Mean costs over the five tasks.

Abstract

Teleoperation remains the default way to collect robot demonstrations, but every episode costs robot and operator time, and every new task starts over. We ask whether a coding agent can do the teleoperator's job at lower cost. In our agentic data farm, DataFarm, the agent receives a data request and one calibration episode, a short recording of the robot in the target scene used only to identify the scene and cameras. It rebuilds the task in simulation, writing the scene, an expert controller, and a success judge. The simulator produces rollouts with exact actions. A video model conditioned on real images renders each rollout as a photographic episode. Because agents write both the expert and the checks that grade it, and the video model may alter the scene, correctness cannot rest on the producer's own judgment. We therefore separate certification from production: the expert is scored by a judge it never sees on held-out initial states, and independent reviewers compare every rendered episode with its simulated state before it enters the dataset. On five real-robot tasks, policies trained only on farm data matched policies trained on the same number of teleoperated episodes (93 vs. 84 successes in 100 trials) and kept 87 successes under an unseen tablecloth and coloured light, where the teleoperation-trained policies fell to 46. Adapting a new task cost the agent $45 on average without human intervention, and for 100 episodes of a task the farm is cheaper than teleoperation whenever a teleoperated episode costs more than $0.71–2.89.

Real-robot results

π0.5 policies trained on 100 episodes from each source, 20 trials per task and condition, on a dual-arm PiPER robot. Sim + DR renders the same simulated trajectories as DataFarm with the simulator and domain randomization, so the gap between them isolates rendering with real images. The farm data mix renders of the original scene with text-varied tabletops and lighting; the test tablecloths appear in no training set.

teleop data Sim + DR DataFarm (ours)

Trials under an unseen tablecloth and coloured light

Neither training set contains the tablecloth or the light. Left: policy trained on teleoperated data. Right: policy trained only on farm data. Real-time, no cuts within a clip.

Teleop data
DataFarm (ours)

The farm

The agent does the teleoperator's job by a different route: it writes the task once, and the farm produces every further episode.

1
Ground the scene, qualify the simulator. From a one-line request and one calibration episode, any short recording of the robot in the scene, the agent aligns the cameras, fits the scale and physical parameters, and labels every quantity as measured, fitted, assumed, or unknown. Skill probes check that the simulator can reproduce the actions the task needs. None of the episode's actions are reused.
2
Write the task and the expert. It writes the task contract, the scene and initial-state sampler, the expert controller, and the success judge. The expert is developed against a judge it cannot see, admitted on held-out seeds, and run from many initial states to produce episodes with exact actions and states.
3
Render the real look. A video model conditioned on the calibration episode's images turns each simulated rollout into photographic multi-view video. The simulator owns actions and states; the renderer may change only appearance.
4
Review and harvest. Independent reviewers compare every decoded episode with its simulated state. Only passing episodes enter the dataset; every attempt and failure is kept.
Overview of DataFarm on the pull-drawer task
Overview of DataFarm. A one-line request and one calibration episode of the scene are the only inputs; the agent builds the field, grows the real look, and harvests a reviewed dataset.
The DataFarm pipeline on one drawer episode
The pipeline, followed on one drawer episode. Each stage is signed by a component other than its producer; a reusable capability layer of skills, role instructions, render templates, verifiers, and harness is versioned apart from the task artifacts the agent writes.

The data

Each farm episode is a simulator rollout re-rendered as photographic video: same actions, same objects, same contacts. Left and right are aligned frame by frame.

SimulatorDataFarm render

New tabletops and lighting

Rollouts rendered with appearance that differs from the calibration scene; the inset is the simulator video of the same rollout.

Warm lamp lightPull drawer · 2×
Blue feltStack cups · 2.4×
Marble tableBeat drum · 1.5×
Brushed steelWeigh apple · 1×

Real, simulated, domain-randomized, and farm episodes

Real, simulator, Sim + DR, and farm frames for five tasks
Real, simulated, and farm episodes at the same task phase (head view). Sim + DR adds domain-randomized textures and lighting to the simulator frame, and ours renders the simulator frame. Real and simulated frames come from different episodes, so object placements differ. Kernel distance (KID, ×103) of head-view frames to real camera frames: 0.83–0.88 for farm renders, against 1.93 for the simulator and 2.50 with domain randomization.

Do renders keep the simulated motion?

Action labels stay valid only if the video shows the motion the simulator executed. We compare the optical flow of each render with that of its untextured simulator video.

Optical-flow error and agreement between renders and the simulator video, per task
Averaged over the five tasks, farm renders deviate from the simulated flow by 4.4 pixels and agree on 65% of moving pixels, close to Sim + DR (3.4 pixels, 73%), which has exactly the simulated motion. The same renders paired with the simulator video 0.5 s later reach 8.8 pixels and 41%, so the measure separates matched from mismatched motion.

Objects that are hard to simulate

Lit scale display, drum, and towel: simulator, farm render, and real
Crops of a simulator frame (top) and of the farm episode rendered from that frame (middle), with the same object in a real recording at the same task phase (bottom). The simulator shows the scale's display as a flat texture, the drum as a red cylinder, and the towel as smooth particle cloth; the farm episode shows a lit display with the simulated reading, a lacquered drum with brass tacks, and the weave of the real towel. The look comes from the calibration episode, set once per task by a render contract costing $4.1–6.2.

Cost

Teleoperation pays for every episode. The farm pays once per task, then only for simulation, rendering, and review.

One-time, per task
$45
mean agent API cost to adapt a task ($38–57 over five tasks), with no human intervention.
Per farm episode
$1.27
mean marginal cost of an admitted episode (rendering, re-rendering, review), the same for a new scene.
Cheaper than teleoperation
> $1.74 per episode
teleoperation cost above which the farm is cheaper, for 100 episodes of a task (mean; $0.71–2.89 over five tasks).
Teleoperation cost per episode above which the farm is cheaper, vs. number of episodes
When is the farm cheaper? Teleoperation cost per episode above which the farm is cheaper, for N episodes of one task; it falls towards the farm's marginal cost as the one-time cost is spread over more episodes.

BibTeX

@article{zhang2026datafarm,
  title={PyRUA-DataFarm: GPT-6 Astra as a Cost-Efficient Teleoperator},
  author={Zhang, Kewei and Ma, Juncheng and Lv, Guodong and Zhang, Hongyu and Liu, Yichen and Hou, Jiaqi and Huang, Ye and Zhang, Chuanrui and Chen, Yuxuan and Si, Ruiyang and Huang, Wenbo and Gong, Shukai and Ruan, Feixiang and Zhang, Yintianrun and Zhai, Xuanran and Deng, Yufan and Xie, Enze and Wang, Duomin and Zhou, Daquan},
  journal={arXiv preprint},
  year={2026}
}