Robot learning typically involves collecting data, training strategies, and redeploying models. RoboICL takes a different path: freezes GPT-6 Astra, does not train task-specific VLAs, provides models with only a small amount of demonstration when reasoned, and keeps track of what the robot has just executed and what has happened to the environment. The current project page gives full results of 30 RoboDojo missions, as well as 929 playable, location-checked triangular execution records.
RoboICL: Embodied In-Context Learning with GPT-6 Astra
arXiv：
The main page of the project:https://mosi-ai.github.io/RoboICL-GPT6-Astra.github.io/
The code is:https://github.com/Mosi-AI/RoboICL
If the model parameters are not trained and the robot is only shown a demonstration of the same task, can it learn precise insertion, long-arm collaboration and depending on the initial state of operation?
This is exactly the question that RoboICL wants to answer.
General visual language models have been able to recognize objects, understand natural language instructions, and demonstrate strong reasoning abilities in open operations. But robot controls not only require the mouse to know what to do, but also require the model to convert the intent into continuous action and continuously modify it according to touch, grab failure, and environmental changes. Once a seemingly reasonable grab may not cling to an object, a trajectory may be interrupted beforehand by the controller, and the second half of a long-distance mission may need to revisit the original target state.
RoboICL's core idea is straightforward: to organize both the demonstration and the robot's own execution history into a model-readable context, allowing the frozen GPT-6 Astra to reference these experiences in each decision, directly producing the next pair of arm moves.
30 tasks: Open zero sample, with the rest of the categories using only one sample
RoboICL evaluated 30 two-armed robo-dojo operations. The tasks were divided into four categories: Open, Memory, Precision, and Long-Horizon: The Open mission had no available task-specific training samples, so zero samples were used; the remaining three categories were unified using one sample.
RoboProbe is the result of a GPT-6 Astra zero-shot by the official robodojo team. PhysicalRSI, Simate-beta and VPP2-Preview are the top three on the latest list.
In the comparison method included on the current project page, RoboICL's 30-task Overall is 50.64, higher than the updated strongest comparison method PhysicalRSI's 39.56. Compared to the official zero-sample GPT-6 Astra controller RoboProbe, RoboICL improved about 2027 progress-score points in four categories of tasks, respectively:
The progress score here is a phase assignment score given by RoboDojo's native scorer, in the range of 0×100. It can distinguish between a complete non-progressive assignment that has completed several sub-target assignments and a complete success assignment, so it cannot be directly equivalent to the task success rate. Overall in the paper is a non-weighted average of 30 assignments per value.
The results of the Open class are particularly noteworthy. There is no expert demonstration here, and RoboICL relies solely on task descriptions, current visual status and the interactive memory the robot has accumulated in this episode. Its 54.75 points describe how to save and organize online experiences, which in itself can significantly change the control of the same frozen model.
Not only a public average: Article 929 execution records can be viewed by layout
Robot papers often use a table summarizing the results, but the mean cannot answer many key questions: did the model go wrong from the start, or failed at the contact stage? did it detect errors and retrieve them?
To this end, the project page has released 929 playable records of the current 30 tasks, covering all 30 tasks. Of these, 880 are part of the full evaluation plan, and 49 are part of the reference trajectory for partial evaluation.
In a web page, you can select task classes first, then task and layout, and see in the same pane:
Three parallel points of view of the left wrist, head and right wrist;
The official score of the layout, the status of the results and the evaluation protocol;
GPT-6 Astra execution instructions aligned with video progress;
Play, pause, go back to the start and 0.5 x 4 x double control.
What RoboICL has done: Get TRAIN and LIVE to use the same interactive grammar
RoboICL does not train new robotic base models, nor does it allow another VLA to generate motion suggestions. The learning component in the system is the frozen GPT-6 Astra, an external control framework responsible for preparing the context, calibrating the motion format, performing inversion motion and returning controller feedback.
The experience seen by the model is divided into two parts:
Demonstration context: The recorded trajectory from the same mission remains unchanged throughout an episode.
Interaction memory: The current rollout from the robot is updated as the action is executed.
The two types of experiments use the same observation → action → execution of feedback → result observation of feedback syntax. The feedback in the demonstration describes the recorded range of expert actions; the online feedback tells the model how many actions were actually executed, which actions were not executed, whether controller interruptions occurred, and new observations after execution.
This design solves a often overlooked problem: the actions of the model are not the same as those actually performed by the robot. If only the original plans of the model are preserved, subsequent decisions may be made in the wrong state; after the execution receipt is added, the model can match the actions with the actual consequences.
In RoboDojo, each observation of three 640×480 RGB images from the left wrist, head and right wrist spliced horizontally into a 1920×480 triptych while providing the state of the robot's core. GPT-6 Astra produces an H×14 motion matrix through a constrained Act interface, describing the incremental decarbonisation of the two arms with a claw target.
Long missions can't just remember the last few steps.
If all historical images are preserved in their entirety, the request will be lengthened as the episode progresses, quickly exceeding the multi-mode context budget; if only the last few steps are retained, the model will forget the initial status of the task and earlier critical operations.
RoboICL uses bounded anchored memory: fixed to retain the initial interaction, set several nodes throughout the task timeline, while retaining the most recent completed interaction. If content is omitted between the two remaining segments, the system inserts a clear <TRAJECTORY_GAP> to avoid mistaken models assuming that both occur continuously.
The model will not be embedded in the context of the whole item, but will extract non-overlapping movement blocks from different stages of the trajectory. Thus, the model can simultaneously see the procedural information of the task at the beginning, middle and end stages.
The paper made two sets of comparisons to this part. The single sample set a fixed total budget of J + B = 24, where J is the number of sample blocks, and B is the number of memory slots interacting. The four assignments (8, 16) ↓ 12, 12) ↓ 16, 8) ↓ 22, 2 respectively were 57.50 ↓ 80.25 ↓ 80.00 ↓ 60.75 ↓ The sample sat the closest, leaving only two online memory slots, and the performance decreased significantly. The main experiment therefore adopted a balanced J = B = 12.
In the same sample configuration, the pinpoint memory is 80.25, with only the first and most recent scheduling being 73.50. The difference is mainly from Build Tower, Put Bottles into Dustbin and Insert Tubes, indicating that long-distance tasks require both up-to-date feedback and early visual evidence.
Deposit Coin: From repeatedly dropping to a millimetre level
Deposit Coin requires the robot to grab a thin coin, place a narrow slot on the safe deposit box, and return the arms to their original position after insertion. In the current 50-layout results, RoboICL scored an average score of 78.00, of which 37 tracks were filled; the second highest method on the current project page, Physical RSI, is 62.27.
The open layout 0 trajectory shows how the model uses the latest wrist angle modification operations. Several previous drawings did not hold the coin steadily, the model changed the cover position and grabbing height; at step 205, it judges from the right wrist screen that the slot is located on the right side of the coin, first making horizontal movements and deviation angles.
This is not a case of rearranging the sample. The sample provides the precedent of the task process and action, while the current observation and interaction memory is responsible for handling the grappling failures, object deviations and insertion errors in this layout.
Ten-mission contrast with GPT-as-Policy: approaching mixed systems only with GPT-6 Astra
The paper also retains a separate set of ten task comparisons, each requiring five layouts. The panel is used to compare public results with GPT-as-Policy, and the scale of the assessment is different from the 30-task master list, so the two cannot be confused into the same experiment.
One demonstration of RoboICL was 22.10 points higher than Direct and 2.00 points lower than the π0.5 + GPT-6 Astra mix. The difference was that RoboICL did not use the learned VLA motion proposal, and the motion was generated directly by GPT-6 Astra via a single interface.
Build Tower is an intuitive representation of the role of the demonstration: in this five-dimensional panel, RoboICL is upgraded from 16 samples of zero to a sample of 100. After expanding to 50 layouts, a sample in the current main list is 80.60, indicating that the small panel results cannot replace the assessment of a broader layout, but the improvement does not only appear on individual samples.
From simulation to real mechanical arm: three samples averaging 78.89
The study also tested Peg in Hole, Folding Towel and Building Bridge on Franka Research 3 single-armed robots. Each mission used GELLO to collect three human samples from the wrist D405 and external D515 cameras; zero samples, one sample and three sample conditions were randomized five times each.
The average progress score of three real robot missions increased from 14.45 on a sample of zero to 63.33 on a sample of three, reaching 78.89 further. All three missions showed a trend of 0-shot < 1-shot < 3-shot. Two-fifths of the three model conditions of Peg in Hole completed the insertion; Building Bridge increased from one sample to three samples by 26.67 points.
The paper also tested the generalization of the actual exceptions to the folding of towels. Three sample models scored 100 on a towel seen, 90 on an unseen towel, but dropped to 40 after replacing a larger unseen towel. This result showed that the demonstration was able to reduce the folding knowledge of the general model to specific robots and operating spaces, while also showing that the model may be exaggerated along the scale of movement in the demonstration.
The above context is long, but most of the forewords are reusable
The paper selected Build Tower, Classify Objects and Put Bottles into Dustbin as a zero-sample/one-sample pairing of 30 episodes, stating token, model calls, API time and end-to-end time.
In 1,810 completed model requests, input tokens amounted to 1.25.1 billion, of which 113.15 billion hits were cached, and token overload KV-cache hit rate was 92.4%.[3] This is due to the stability of the request structure: task instructions, demonstrations, camera labels, and fixed focal points form a stable forefront, with only the latest interactions and gaps nearby continuing to change.
The paper tested optional Jev action duplicate gate controls on two other development tasks. It did not generate action, only judging whether to continue to execute the GPT-6 Astra actions proposed but not yet executed. The GPT-6 Astra requests for align blocks decreased from 181 to 121, and General Pickup decreased from 116 to 60, reducing by about 33% and 48% respectively. However, the success rate affected dependency tasks: General Pickup increased from 4/5 to 5/5, and Align Blocks decreased from 3/5 to 2/5.[4] This section was therefore defined as an accelerated development cluster experiment, rather than an already solved general solution.
What does this work reveal?
The results of RoboICL support a specific judgment: for a general multi-modal model with visual understanding and the ability to generate numerical actions, the value of a small sample is not only to give the correct answer to the puzzle, but also to provide an executable process structure; whereas the robot's own online experience must be preserved in the form of a pair of actions and real consequences.
It also has clear boundaries:
The training data for different methods using seed 0, like the results of Allah, are not exactly the same as the evaluation protocol;
Progress score is a measure of progressive progress that cannot be considered as a complete success rate.
A real robot experiment has only three missions, each set to five trials.
The current validation around the GPT-6 Astra is ongoing, and further experimentation is still needed for the versatility of cross-model, cross-robot formats.
The paper and project pages have provided a more complete chain of evidence than a single average release, with 30-task category results explaining coverage, ten task boards giving controls to direct controls and mixed systems, real-world robot experimental testing across embodiments, and 929 episodes allowing readers to see directly what robots are doing in each layout.
When the demonstration and execution history are organized into the same language, robots can use a small amount of experience adjustment behavior in the previous context without having to wait for the next round of parameters to be updated. RoboICL is not a new robot pre-training model, but a method to continue learning the existing general model as it is deployed.
The main page of the project:https://mosi-ai.github.io/RoboICL-GPT6-Astra.github.io/
The paper:
The code is:https://github.com/Mosi-AI/RoboICL
Information from the author
The first three authors, Xiamen, Shenzhen, and Shenanda, who are researchers at Samsung's Big Model, graduated from Beijing University, Tsinghua University, and the Institute of Automation of the Chinese Academy of Sciences, respectively, and were members of the core team of the leading Big Model Manufacturers and First Line Manufacturers, are now focusing on Scaling Law-driven Big Model mass training to push model capabilities from the digital world to the physical world.
Communication author Dr. Tsai Chen, current head of Samsung's Big Model, youngest strategic business head at Samsung, is the overall head of Big Model Training. According to public information, he has organized and led AI MODEL TF, aimed at the Samsung global ecosystem, responsible for basic model training and core technology research from the digital world to the physical world.