All field notes

Introducing Hebbian RUA

Up until recently, getting a robot to do something new meant one of two things: writing control code for it line by line, or training an AI using simulation and human demonstrations.

A third path is increasingly promising. [1] Frontier multimodal models can now look at a camera feed and understand the scene, write and run code against a robot's controller, [2] [3] and check their own progress on the task, retrying when something goes wrong. [4] [5] [6] [7]

Put these pieces together and the way you interact with a robot completely changes. You describe the task in plain language, the way you would to a coworker, and agents work out the rest.

The emerging term for this is Robot Use Agents (RUA). [8] Computer Use Agents enable AI to operate software. RUAs let it operate robots in the physical world.

Once telling a robot what to do is as simple as a conversation, the hope is that the cost of programming robots for autonomous work drops from months to days.

Much like what coding agents have done for software, we envision this unlock leading to an abundance of diverse robot use cases, especially ones that were previously cost-prohibitive.

At Hebbian, we have been working with research and industry partners to bring such capabilities to the masses.

Today, we're introducing Hebbian RUA in public beta. With RUA, you give a frontier model a goal, camera views, and a set of predefined robot tools, and it controls a robot to perform the task.

Think of RUA as what frontier models need to understand how to control a robot. Developers then bring their own model and loop, based on their task.

Agents can do much more than control arms based on what they see. [9] [10] We're excited to share what we're seeing work, and where there's still room for improvement, in the coming weeks.

There is still a lot to learn and discover, and we would love for you to build with us.

References

[1] Ahn, M. et al. (2022). Do As I Can, Not As I Say: Grounding Language in Robotic Affordances.

[2] Liang, J. et al. (2023). Code as Policies: Language Model Programs for Embodied Control.

[3] NVIDIA (2026). ASPIRE: agentic robot skill discovery.

[4] Anthropic (2026). Claude plays robotics.

[5] Google (2026). Gemini 4 Argon for Robotics.

[6] Robocurve (2026). GPT-6 Astra on robotic manipulation.

[7] NVIDIA (2026). Astranomy: studying Astra as a robot policy.

[8] Isola, P. (2026). Robot-Use Agents.

[9] Huang, R. et al. (2026). RAX: Scaling Online Exploration for Robot Agents.

[10] NVIDIA (2026). ENPIRE: robot policy improvement through physical experiments.