It's the first time in the history of the world.
AI has conquered all humans by doing mathematical problems and writing code.
However, the same set of models get the CAD and simulation software engineers use every day, giving each task a maximum of 24 hours to design the curve axis, to give memory wiring, to calculate the torque spreadsheet, and the best GPT-6 Astra is only 37%.
The rest of the models in this evaluation are almost entirely tied to zero.
Xplore AI released RWE-Bench, fully titled Real World Engineering Bench, on October 27.
There are 20 questions, 9 mechanical questions, 6 aerospace questions, and 5 electronic questions.
Each question is asked by an engineer in the team, who also has to make a reference answer using the same set of tools.
Judging is neither AI nor AI.
The program calls for the design to be checked on a case-by-case basis by the physical solver, even if one of them fails to meet the standard.
In terms of mechanics, the theme is the curved axis of the design of the three-stroke diesel engine.
The technical summary lists the quality limits, stress limits, fatigue coefficients and curve displacement variations, each with specific numerical values.
Adding more weight would be overweight, reducing turnover and imbalance, as well as completing millions of cycles at 2400 turnover loads.
Only CadQuery, Gmsh, CalculiX, and Python are in the hands of the model, modeling, mapping, and finite metadata analysis.
The electronic puzzle was designed to complement DDR3 on a mass-produced FPGA board.
Nearly 300 elements have been laid on the board, and the other lines are all woven and unmoving.
The remaining more than seventy networks have to run through the gaps of eight layers of the board, and the timing bias and resistance are limited in hardness, so long as one superblock, the entire board is not counted.
The aeronautical problem is to calculate the heat transfer of the high-pressure turbine to the leaf, from the construction of the pipeline to the convergence of the CFD, and finally the exchange of the heat coefficient and the total heat load.
In engineering companies, the threshold is quite high for those who have years of experience and are familiar with the software to do so.
Data outside the scores1. First look at the rankings.
GPT-6 Astra 37%, Claude Opus 5.5 29%, GPT-6.1 Sol 25%, Gemini 4 Argon 12%, Claude Fable 5.1 7%, and the rest of the models in this rating are almost completely zero.
2. In addition, the response time ratio is better at explaining the problem.。
Each question takes a maximum of 24 hours, but each model takes an average of two to three hours to complete, with the longest taking only about 4.6 hours.
Human engineers encounter this problem by repeatedly modifying models and running simulations for more than this amount of time.
The model probably thinks it's done, but it's wrong...
3. Spending more money doesn't have to be high.
The GPT-6.1 Sol runs at $10 each, while the Astra and Opus 5.5 run at $58, with scores only 12 and 4 respectively.
Claude Fable 5.1 is close to $98 at a time, the most expensive of all, with an approval rating of 7%.
4. If each question is allowed to be tested five times, even once, the score of Opus 5.5 can reach 70%, 60% higher than that of Astra.
There are also questions to be asked on the list.1. There are only 20 tracks and the sample capacity is very small.
Officially given 95% confidence intervals, Astra is 30.7% to 43.8%, Opus 5.5 is 21.9% to 37.6%, and the two sections are overlapping.
On page 2, it says that all models have the highest reasoning intensity, but in the data file, the Gemini 4 Argon is high, not max.
Of course, the newly released flagship Gemini 4 Carbon did not appear in this review, and we look forward to Xplore AI adding to it.
3.Xplore itself is selling RL training environments.
The official website states that they can help model companies to evaluate, build training environments, but the topics in the list are not authorized to train.
The lower the scores on the list, the better the business will sell.
This doesn't mean that the data is problematic, that the topics and the way they are judged are public, but that controlling the difficulty of the topics is good and bad for their business.
4. There are four ways in which all models have not been made, and which ones are not officially made public.
The next step for Xplore is to include topics in biomechanics, civil and materials engineering, where engineers and researchers are also being recruited.
When the SWE-bench was first released in 2023, the best model solved less than 2% of the problems, and then the scores turned into what everyone saw.
Please look forward to the rising scores of the new models on the RWE-Bench.