It's the first time in the history of the world.
What's left after AI completes its mission?
A team of researchers from Beijing Aerospace University, the Chinese University of Hong Kong, and the National University of Singapore recently released OmniHarness, which offers a path: let the visual intelligentsia actively practice, check out the results, and keep the effective flow.
In the face of the next demand, AI has more than one verified method.
The full text of the paper:
The main page of the project:https://omniharness.github.io/
Code and description of operation:https://github.com/OmniHarness/OmniHarness
95% of creative task solving
In the ComfyBench creative task, OmniHarness reached 95%, exceeding the best control of SymbOmni in the table.It was 27.5 percent.The overall resolution rate for Codex GPT-4o configuration is 92.5%.
Pass in the table to see if the process can be executed and Resolve to see if the output can meet the task requirements.
Passes with both OmniHarness configurations are 100% and ComfyMind and SymbOmni have done the same.How many qualifying results did the process yield?
The reasoning skeleton is also set at GPT-4o, with OmniHarness at 89.5% overall, ComfyMind at 83%; and creative tasks at 95% and 57.5% respectively.
This makes how experience is summoned, how tools are organized, a factor worth observing beyond model ability.
After adopting a codex plan, the overall89.5% up to 92.5%Complex tasks increased from 76.7% to 83.3%. Different scheduling configurations correspond to different multi-step tasks performance.
The advantages are also wide-ranging: 83.3% of complex tasks still do not exceed 85% of ComfyMind, and not all sub-sets lead.
What is the picture of these results?It's easy to get into a flower graffiti.
The graffiti was redrawn to write true red flowers, and its style was used to produce a flower field.
Two generations are connected by an intermediate image.
Instead of four-dimensional cartoons, the task becomes to tell the whole story on a Sunday morning: wake up, look at your phone, find it's Sunday, keep lying flat.。
In exchange for posters and book covers, the theme, title and edition form form a set of requirements that must be met simultaneously.
The editing of the table is more subtle: the fork should be removed, or replaced with a fork, and the food, cups and table layout next to it should be kept as much as possible.
If you want to change your goals, you have to consider your surroundings.
How do you keep the experience after you've done it?The need for creativity is constantly changing. Red flowers may be replaced with other subjects, and the layout may need to be adjusted after repainting.
OmniHarness uses symbol strategies to learn.To organize successful verification processes into strategies that can be used for a class of tasks.
Understanding the task of flowering: flowers and colors can be replaced by redrawing a drawing, then using a reference to create a new image as a way of connecting, can be an adjustable method.
The conservation strategy includesWorkflow templatesExample inputs such as applicable conditions and resource dependency, current images, specific descriptions are extracted.
Subsequent calls are made to first check if applicable, then bind new inputs, adjust steps or combine multiple strategies as requested.
Submitting, checking, accumulatingHow did the experience change?
The first step is to choose where the exercise is worth exploring.
Self-exercise takes place before the downstream mission is clear.
It is difficult to fill in the gaps by simply doing familiar tasks; challenges are too remote and there is a lack of methodological support.It's a great way to explore.
By default, each round generates 10 candidate questions to determine which one to practice by multiplying the new flexibility and capacity boundary scores:
This round of candidate missions comprises a set.
Measuring whetherIt's a little practice.It reflects the current capacity to provide moderate support for exploration.
The more people who practice in similar situations, the lower the scores of novelty:
It contains the capabilities required for the task, and records the number of exercises performed by the respective task manager.
The scores are averaged according to the required ability, avoiding a lot of ability items.
Each capability is supported by the highest reliability of the applicable, uninterrupted process;Ninety-five percent of Wilson believes in the bottom line.It's estimated.
The ability of the whole question to estimate the minimum value is:
This is the equivalent of looking at the shortcut: most of the steps are skilled, and a lack of reliable methodology will still drag the whole process to completion.
Then the ability estimate is converted into the test scores:
When the score is highest; when the estimated value approaches 0 or 1, the score decreases and the practice deviates.It's a mission that has foundations, but still challenges.
The degree of support for expressive abilities, which is a selectionally inspired score.
The second step is to keep the check up to date with each execution.
The system arranges the steps and dependencies, compiles the code into the ComfyUI workflow, and checks that the process is running, that each step has an effect, and that the end goal has been achieved.
For example, if a red flower has deviated from the requirement, the next step can be followed, and the deviation can be entered into the new image.
After the failure, the system positioning problem and local fix, as much as possible retaining the verified parts; request sub-intelligent assistance when necessary, and check again within the re-test budget.
The third step is to make success the basis for the next step.
Successful verification processes are abstracted into the library and failures record evidence, causes and correction methods. Equation processes combine, reliability with call updates, and experience continues to influence selection and execution.
The strategy will also be subject to follow-up testing. New success records will continue to affect its reliability and call priorities.
Experience with a different userDo you still use it?
The team froze the strategy library after self-exercisingThe host retains the original control process, and the model is not tailored.
The content delivered has both successful process templates and failed proofs and remedies. Other intelligent entities use them through their own retrieval, planning and execution mechanisms.
The same experience has improved the overall resolution of all three systems:
ComfyAgent increased from 32.5% to 57.0%
ComfyMind increased from 83.0% to 88.0%.
SymbOmni has dropped from 86.0% to 89.0%.
The corresponding creative tasks increased by 27.5, 17.5 and 10 percentage points respectively.
The improvement of creative tasks exceeds their overall increase. In the face of new requirements, processes and experience in correcting errors can still help different systems.
There is also a layer of migration: a freezing strategy library of only image tasks, used for video workflow.
The overall video resolution rate is85.9%higher than the best comparisons in the table5.1 pointsVideo to video reached 80.0%, up 13.3 percentage points.
Among them, image strategies are responsible for content construction, reference maintenance and other operations, while sequencing is still handed over to video-specific components, both of which must be adapted and combined.
End of a missionThe use of experience has just begun
This path also requires computational costs: autonomous exercises, multiple rounds of verification and repair will increase costs, and there is still room for optimisation of recovery efficiency and budget reasoning.
OmniHarness demonstrates a method of accumulation: model parameters remain unchanged, and methods can be invoked with practice updates.
The experience left behind today is an opportunity to be the starting point for tomorrow's new needs.