After the decision-training language model reopens the generating switch, it is not necessary to be able to reason steadily. In the researchers' experiments, the 4B model barely generates reasoning on about 40% of the problems; the 9B model moves to the other end, continuing to generate on about 70% of the problems until it reaches the length limit.
These two opposing failure patterns suggest that the decision model should move from "direct judgement" to "step-by-step computation" rather than a one-time switching operation. In addition to choosing the right answer, the model needs to form an effective reasoning process that ends when the computation is completed.
Researchers from StartLux, the University of Hong Kong and the South China University of Technology have proposed StartLux-Decision around this issue. The study builds on a single prediction version of Instant and the on-demand version of AutoThink, which builds on a unified option probability interface and develops two types of capabilities through three stages of training, and proposes RLCD2, a change-training reasoning strategy to reason on the quality of predictive probability.
StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
Link to the paper:https://arxiv.org/abs/2610.17309
The code link:https://github.com/StartLuxLabs/StartLux-Decision
Model weight:https://huggingface.co/startlux-models
Unified Probability Interface, two focused designs
Requests for routing, retrieval of results judgments, tool calls for proofreading and task acceptance usually do not require a model to generate an explanation, but instead require it to make judgments between specified candidates. StartLux-Decision defines such requests as "Input status + question + candidate" and outputs the probability of each option, supporting multiple options, non-judgments and grade scores.
Probability comes directly from the existing output layers of the language model. The researchers mapped the options to letters such as A, B, C, and read the corresponding logits at the last location of the cue, with temperature scales and softmax distributed in probability, without adding any additional headings. Instant's single prediction, as well as the two predictions before and after AutoThink's inference, used the same reading method.
The difference between the two versions is first seen in the attention structure. Instant does not need to self-return generation, and full input has been obtained before the forecast, so the researchers removed the causal mask in the Full Attention layer, allowing the token to interact in two directions within the prompt; the Linear Attention layer still retains the causal computation.
The changes here are not just the last position to see what information. The final reading position of the causal model is to see the full hint; the bi-directional further allows for a description of the state ahead, integrating the background problem with the candidate in the middle layer. This provides a different representation of the decision task that requires a re-understanding of the input of the combined options.
Figure 1 shows a controlled comparison of two-way attention. The left side only modifies the direction of the prompt attention of the Full Attention layer, while the Linear Attention layer still maintains the causal calculation; the right side shows the same data, the difference in scores at the same initial checkpoint.
This modification was subjected to control checks on the same data as the initial inspection point. On the five parametric scales, four DI 0.3 open-section scores were improved, with a maximum increase of 2.4 points; the variation of 2B was −0.2 points, with no similar gains. At the same time, the paper reported that there was a 0.2 ton 0.7 point fluctuation between different random seeds under the same training configuration. Thus, experiments supporting binary attention were effective in most settings, not necessarily for all scales.
Shared decision-making capabilities are built up from the first two training stages. In the first stage, the language model is fully parametrically monitored, with 0.8B9B using about 2.7 million decision samples, 27B using about 800,000; in the second stage, about 1.7 million samples are trained and combined with adapters, while the focus direction is differentiated. Instantly, a single prediction model is formed, AutoThink then enters the reasoning monitoring and reinforcement learning stage.
Figure 2 - Training and reasoning processes. The first two stages establish decision-making capacity and differentiate attention design; the third stage introduces AutoThink to reasoning supervision with RLCD2. Instant and AutoThink use a common option probability reading format, but not the same release weight group.
Read-or-Think: Predict and decide if it's reasonable
AutoThink calculates the initial option distribution p0 with a cause-and-effect pre-calculation. According to the paper's default rating configuration, the maximum probability of an option returning directly is 0.9; when less than 0.9, the model re-processes the prompt, generates a reasoning trajectory, and then reads the option distribution p1 from the end of the trajectory.
After entering the reasoning branch, the final output does not fully cover the initial forecast, but is weighted on two distributions:
Seventy-five percent of the weighted after-reason selections based on the Brier scores on the training distribution reservation sample focused on the predictive quality of the entire probability distribution rather than just the correctness of the highest probability option. This weighting makes the score optimal on 9B and 27B, and close to optimal on 4B.
The paper illustrates this process with two examples under the same data export policy. The policy states that data that are in a state of legal preservation usually need to be audited manually, but the rule does not apply when less than 100 records are exported; other rules require that less than two exports have been made in the past 30 days and that less than 500 items can be approved. Both cases do not trigger a higher-priority rule of rejection.
The first case filed for an export of 3,332 records, which have been exported once in the last 30 days. The model once predicted a probability of "artificial transfer audits" of about 0.96, exceeding the threshold, so it responded directly.
The second case only exported 8 records, which have not been exported for the past 30 days. The initial model is still biased towards artificial auditing, but the highest probability is only about 0.57, so it entered reasoning. It validated the "less than 100" exception, then checked the approval conditions and concluded with 159 tokens. The probability of "approval" after reasoning is about 0.995, with the initial prediction merging about 0.848, and the final option changes with it.
The following statistics cover 7,800 core questions: about 39% answered directly, about 61% entered reasoning. This chart reports the accuracy of the single question, not the completion of the subsequent text.
This segmentation is also reflected in the overall statistics. In the 7,800-way Decision Pivot core questions, 27B AutoThink responded directly to about 39% of the questions, with a single question accuracy of about 96%; the remaining about 61% went into reasoning, and the accuracy after the merger increased from about 57% to about 93%.
RLCD2: Reasoning for improved evaluation using probability predictions
The problem of when triggering reasoning is solved by trust distribution, but it is not guaranteed that the triggered reasoning is effective. For the above-mentioned empty reasoning and non-termination phenomena, the third stage involves the use of some 4,000,000,000,000-ton-21,000-ton reasoning supervised sample training adapters to restore the reasoning generation and termination capabilities of scale models, and reinforcement learning.
The researchers proposed RLCD2, known as Reinforcement Learning for Calibrated Deliberative Decisions, to explicitly optimize the contribution of reasoning to decision quality along the lines of probability decision training explored by RLCD and OpenJev-RLCD.
It differs from training that only rewards the correct answer: selecting the correct answer in the same way, giving it a probability of 0.55 or 0.95, will affect whether the downstream system automatically executes, continues testing, or is handed over to man-made processing. Therefore, rewards require evaluating the full probability distribution, rather than only looking at the maximum probability options.
For this purpose, the researchers froze a model that had been supervised by reasoning, and used it to read the probabilities of the pre-reasoning and post-reasoning options separately. The models that generated the reasoning continued to train and froze the copies responsible for scoring.
Both probability readings here come from a frozen scoring model, t as the target distribution, ð as the reasoning trajectory generated, and L as the token number of the trajectory. S uses strictly appropriate scoring rules for decision weighting, while maintaining the "probability of real reporting results with optimal scores", while giving greater weight to probability errors in high confidence regions. It does not simply encourage models to become more confident.
In the release configuration, the soft-length budget B is 3,072 tokens, more than the reward of 10−4 deductions per token. This budget is not a hardcut: the model can still continue to be generated until the maximum of 4,096 tokens. The training thus establishes a constraint between probability quality and computation costs, without requiring all the problems to be completed in the same short stretch.
What skills have actually changed after the shortening of reasoning?
In a reserved sample of 2,986 subjects, the researchers compared the reasoning supervised model with the RLCD2-trained model. For isolation reinforcement learning, the experiment enabled reasoning for each question, using a unified upper generation limit of 4,096 tokens; it differed from the previous method of dependable distribution.
9B's change is most noticeable: the ratio of self-termination of reasoning increased from 45.9% to 93.9%, the average length decreased from 2,494 to 868 tokens, a reduction of about 65%; 4B's self-termination ratio increased from 89.5% to 99.3%, the average length decreased from 1,140 to 722 tokens.
Figure 4 shows the termination before and after the RLCD2 changes in length and quality. The first four columns enable case-by-case reasoning on 2,986 reserved samples; the right column uses a 0.9 gate control on Decision Pivot. 3,072 is a soft punishment budget, generating a maximum of 4,096 tokens.
The final decision accuracy on all three scales in this group of reservation samples, with variation ranges containing zero. Therefore, the most directly supported conclusion of the controlling group is that reasoning termination and generation efficiency are improved, rather than reinforcement learning significantly improving the accuracy of all scales.
Shorter is not always better. On DecisionPivot, the 4B model reinforces a pre-apprenticeship accuracy of 60.4%; when the length penalty comes into effect after 512 tokens, the indicator drops to 52.9%. After loosening the soft budget to 3,072 tokens, the accuracy returns to 59.9%.[3]
Completely eliminating the length penalty, while achieving an accuracy rate of 61.7%, left about one-third of the reasoning of the assessment unable to end on its own within the generating ceiling. The experiment revealed a specific leverage: the need to suppress repetitive generation, but not to compress the problems that really require long computation. The release model uniquely adopted a soft budget of 3,072 tokens, which is the configuration under the leverage, rather than the optimal length for all tasks.
Decision Pivot: The key details have changed, and will the answers change with it?
The average accuracy does not necessarily reflect whether the model recognizes the conditions for determining the answer. A model may be correct in most common situations, but err on rule exceptions, threshold boundaries, or a modified field.
To test this sensitivity, the research team constructed Decision Pivot. Its core consists of 13 classroom-generating tasks, each of 600 pathways, with a total of 7,800 pathways, covering scanning matches, desk checks, linear conflicts, permission inheritance, target state checks, and tool call checks.
The core indicator is the accuracy ratio: only two questions in the same set of questions are answered correctly, then passed, and then averaged to 13 class tasks. It checks more directly than single-issue statistics, whether the model distinguishes on the surface similar but different decisions.
Figure 5 shows the accuracy of the execution of the decision pivot core task. The real-color portion is Instant, the slope portion is AutoThink relative to Instant; the error line is 95% bootstrap confidence interval.
4B, 9B and 27B AutoThink have an accuracy of 59.9%, 59.9% and 86.9% respectively; the Instant correspondents are 41.5%, 47.5% and 75.4%, and the difference is 18.4%, 12.4 and 11.5 percentage points. In the control system, Decision 3.0 d3 and Perplexity Decider v1.1 are 71.3% and 70.1%, respectively, enabling the GLM-5.3 with low-throughput intensity configuration to be 82.2% respectively.
Increases in 4B and 9B are not uniformly distributed across all tasks. The improvements in 4B and 9B are concentrated on tasks that require computation or individual checks, such as screening, target status verification, and classroom conflict; while in tasks such as policy interpretation, change in terms, and dependency handling, strong single predictive models still perform better. This shows that the value of explicit reasoning is related to the computation required for the task and cannot be summarized with just one summary.
Fixing the same model, separating the benefits of reasoning itself
Instant's comparison with AutoThink also includes differences in the direction of attention and the training phase. To determine exactly how much additional reasoning contributes, the same AutoThink checkpoint needs to be fixed, only to change whether reasoning is enabled.
The researchers scanned the confidence threshold from 0 to 1: the former never reasoned, and the latter always reasoned. 27B AutoThink achieved an accuracy rate of 48.0% when it closed the reasoning on Decision Pivot; the default 0.9 threshold was 86.9%, with an average of 304 reasoning tokens per question. 9B increased from 22.3% to 59.9%, with an average of 1,365 reasoning tokens per question.
Figure 6 Fixed AutoThink checkpoints change the effect of the reasoning thresholds. The top row is reserved for the single-question accuracy of the sample, the bottom row is the Decision Pivot completion of the parity accuracy; the horizontal axis contains zero token spending for the directly answering question, the horizontal axis ranges vary in size, the star is marked with 0.9 thresholds.
Of these, 48.0% were results from AutoThink's own shutdown of reasoning, not 75.4% from 27B Instant. The first group compared the role of explicit reasoning in separating the same model, and the second group compared two complete training programs that cannot be interchanged.
In a reserved sample of trained distributions, 27B's default configuration improved accuracy by about 2.5 percentage points compared to closed reasoning, using an average of 273 reasoning tokens per subject; 9B gained about 1 percentage point at the default threshold, while enabling reasoning for all subjects did not further improve accuracy.
Decision Index: overall scores and task differences
In the open section of DI 0.3, the five-tier Instant model scores from 0.8B to 53.8, respectively rising to 59.1 for 2B, 67.9 for 4B, 70.0 for 9B and 72.8 for 27B. The strongest third-party decision-making model on each scale has a difference of 7.4 times 16.9 points.
Figure 7 shows the scale contrast of the open section of DI 0.3. The purple line is Instant, the gray line connects the strongest third-party decision-making models of all sizes; Jev is the unpublished hosting model of the parametric scale. The scores in the chart do not represent the results of the complete DI 0.3 list.
AutoThink's combined scores further increased to 68.21 for 4B, 70.45 for 9B and 73.37 for 27B. The overall increase was small, but the results in the sub-fields were more reflective of the difference: in "knowledge and reasoning", 4B and 9B were up by about 7.0 and 6.2 points from Instant, respectively; in some other areas, Instant retained the advantage.
These values need to be understood within the scope of the evaluation of the paper. The open part of DI 0.3 corresponds to only 20% of the scores of the complete list, and has been used for the data formula and model selection of this study; it is not an independent test set. The scores of AutoThink are the results obtained by the paper using the open evaluation tool and cannot be considered as its entry into the single prediction list that excludes rational trajectory generation models. The paper therefore also uses Decision Pivot, which was not involved in the development of the selection, for independent testing.
Is probability, after being screened and routed, able to retain its original meaning?
In addition to the model selection of the answer, there is a question that directly affects the use of the system: is an "80% certainty" that still corresponds to the approximate actual correctness rates in different decision-making processes?
The paper found that the probability calibration on a single prediction could not automatically migrate to a downstream strategy. With the 9B model as a verifier, the original prediction was given 80% to 90% confidence in a single answer, with an actual accuracy of 92%; but the accuracy in the same confidence interval decreased to 78% when the highest score was selected from 32 candidate responses. This decreases the accuracy in the fixed confidence interval, and the overall accuracy of the best-of-N cannot be inferred.
The reason for this is that the selection operation changes the conditional distribution of returning predictions. The highest scores not only reflect the quality of the answers, but may also contain upward fluctuations in the scores; the scores of other candidates can also provide information about the difficulty of the problem, which is not fully represented by a final score. This deviation may still exist as the pool of candidates expands, even if a single prediction is normally calibrated first.
Figure 8 - Strategic perception calibration. The left side compares accuracy in the fixed confidence interval after conventional calibration and strategic perception calibration; the right side is the average absolute confidence in the four categories of processes.
The researchers therefore proposed policy-aware calibration: first performing actual screening or routing strategies on labeled calibration data, and then using the returned predictions and related strategy information to match the calibration mapping. Candidate selection requires consideration of information such as the number of candidates, average candidate scores, and on-demand reasoning and ranking will trigger the confidence of the routing to be incorporated into the calibration.
This step does not re-train the underlying model and does not change the correct answer to the original choice of errors, but does adjust the estimate of the reliability of the final output. In a set of 184 sets covering best-of-N, gate controls, hierarchy and pre-selected reasoning, there is confidence in the symbols. The accuracy rate gap from −11.7 to +20.5 percentage points is reduced to −2.7 to +2.1 percentage points.
Reliability is not only a property of model checkpoints, but also related to the number of candidates, route rules and task distributions; after changes in system strategy, calibration and acceptance of thresholds also need to be re-checked.
Testing to separate quality, computational costs and system reliability
The deployment experiment extended this distinction. The paper provided vLLM, llama.cpp and MLX support, and evaluated different quantization formats. Under the single H200, BF16, 256-token prompts and single request conditions, instant's single-reading median delay was 24x52 milliseconds; once AutoThink triggered the inference, end-to-end delay also included trajectory generation and follow-on readings, which could not be performed along single-reading milliseconds. On the Apple M5 Pro with 64GB of unified memory, the 27B BF16 version surpassed available memory, enabling 8-bit and 4-bit chemistry to run locally.
The study also leaves clear open questions. The benefits of new decision-making data largely fall on covered task families, with limited evidence of cross-task migration; a fixed confidence threshold does not equate to finding the best computational allocation strategy for each task; and changes in distribution weaken the calibration mapping that is already adapted.
StartLux-Decision addresses these issues in a testable design: the direction of attention affects the representational ability of a single prediction, the probability gains and the production costs of RLCD2 optimization reasoning, the decision pivot testing for the discernment of critical details, and the strategy perception calibration focuses on the behavior of the probability after entering the system.
© THE