Researchers at MIT and New York University have built a system that lets robots learn the reason behind a typed command. The platform, called FLEX, uses a large language model to turn a single correction into a broader rule.
The study will be presented at the annual Conference on Robot Learning this month. MIT News published details of the work on November 9, 2026.
From One Command to a General Rule
In one simulated test, a robot arm carried a cup of coffee across a cluttered table. A person typed “Don’t spill on the laptop!” while the arm was moving.
The robot avoided the laptop as told. It also steered clear of a nearby phone, even though the person never mentioned it.
That step comes from the language model, which learned from huge amounts of internet text. “People might have written about, ‘Oh, my phone dropped in the water and now it doesn’t work,’” said Nathan Dennler, a postdoc at MIT’s Computer Science and Artificial Intelligence Laboratory.
Dennler said such text teaches the model that liquids and devices do not mix. “So LLMs pick up on the idea that liquids plus devices leads to things not working,” he said.
The team used OpenAI’s GPT-4o as the language model. Dennler said some common sense “gets absorbed into the LLM” because it has trained on text from so many people and places.
A Three-Step Process in Five Seconds
FLEX works in three steps called Add, Generalize and Mask. The full process takes about five seconds.
In the Add step, the robot puts an object named in a command on its list of things to avoid. Earlier methods stopped here, so people had to name every item one by one.
Dennler said this older approach breaks down when the list “becomes super long.” Some items on it may also no longer be in the room.
The Generalize step is the new part. The language model reviews all the items a person has flagged and looks for a pattern, such as a mug pointing to a rule about fragile objects.
In the Mask step, the robot’s camera system checks which objects have left the scene. It then removes those items from its list.
The team tested FLEX in simulations of three tasks: serving coffee, moving a candle and carrying a bowling ball. Volunteers typed commands such as “Stay away from the stack of paper” and “Don’t drop on the mug.”
In each case, the robot followed the literal command. It also learned to avoid electronics, flammable items and fragile objects.
The study was led by Andreea Bobu, an assistant professor of aeronautics and astronautics at MIT, who heads the CLEAR Lab. Co-authors include Audrey Lee of MIT and Ilia Sucholutsky of New York University.
The work was partly funded by the MIT Generative AI Impact Consortium. Support also came from the MIT Schwarzman College of Computing’s Social and Ethical Responsibilities of Computing program.
“Everyone’s homes are different, and everyone cares about different things, so no robot comes out of the box knowing what matters to you,” Bobu said. She added that FLEX can pick that up “from a sentence or two instead of asking you to spell out every rule.”
