EMNLP'26 Oral, MIT: How far can we go from interviews to AI simulations of real people?

  It's the first time in the history of the world.

If you talk to someone long enough, can the big model become their "digital separation"?

From virtual users, synthetic interviewees, to personalized AI agents, an increasingly popular technological route is to get to know a real person through interviews, questionnaires, and profiles, and then have the big model play the person and predict how he'll answer questions he hasn't seen before.

But here's a more difficult question:AIEven after learning what you're thinking now, in a new situation that hasn't been asked before, can it continue to answer like you?

To test this, researchers at institutes such as MIT have proposed the HugAgent, a large model based on real-life interviews with participants to predict their responses in emerging contexts such as public health, public surveillance, and housing density. The study covered 54 participants, 1,742 assessments, and tested models such as GPT, Claude, Gemini, DeepSeek, Lama, Qwen and others.EMNLP 2026 Main Oral。

Link to the paper:https://arxiv.org/abs/2510.15144

The code link:https://github.com/jajamoa/HugAgent

The main page of the project:https://jajamoa.github.io/HugAgent/

Research has found that individual interviews do help models more accurately simulate a specific person.And they talked to each other for a long time.AIHe will be more aware of what he thinks now, but not necessarily more aware of how he will respond when the conditions change.

This also puts "digital separation" in the face of a more fundamental problem:AIAre you building a model about yourself or are you just building an increasingly detailed personal portrait?

From "like a normal person" to "like this specific person".AICan you continue to represent this person?

Over the past few years, simulating humans using large models has become a real research route. Researchers have taken LLM as consumers, voters, and experiment participants even further, and have begun to try to simulate someone who is real: building "digital isolation" through lengthy interviews, questionnaires, and profiles, and allowing AI to continue to represent itself in the new issues that follow.

The more information a person tells an AI, the better the AI knows him, and it seems that the more it should be able to predict his response.

But here's a key leap:

Describing a person is not the same thing as predicting a person.

Suppose a person says in an interview that he supports universal health care but is concerned about cost, efficiency and personal choice.

But what if conditions change?

Would he make the same choice if the new scheme could save his family $3,000 a year, or if private insurance could still be retained?

At this point, the model must not only remember "he supports public health", but also judge from past expressions what he really values and how he will re-evaluate when different conditions change.

This is exactly what HugAgent wants to test:

If we build a persona rich enough by interviewing it can predict how the person will respond outside of the interview.

This is an inevitable issue after "like a person" to "like this person".

In order to test whether AI can simulate "someone", one must first know what the person is thinking.

HugAgent, therefore, did not rely on public figures or web texts to recruit real participants. The team chose three themes that were clearly divisive in the United States and involved real value for money: universal health care, public monitoring, and increased housing density, i.e., the relaxation of partial subdivision limits that allowed for the construction of taller, denser housing.

Step one: The questionnaire, write down "you are now".

Participants first fill out demographic information and then give 1 to 10 points of support for different topics. At the same time, they also need to evaluate the impact of a range of reasons on their positions.

For example, opposed to housing density, one person may be most concerned with the appearance of the community and the value of the property, while the other person may be primarily concerned with transportation, rent or housing supply. The ultimate "support or opposition" is just one answer that really distinguishes between different people, often the reasons behind the answers and how important each of these reasons is.

This information collectively constitutes a person's current belief state, which is his current position and the reasons for supporting those positions.

Step 2: Change the conditions and record "what if"...

Then the questionnaire began to change one of the conditions.

If the housing density increases, local rent decreases from 10% to 15%, if public health can save $3,000 a year for a family, if public surveillance videos only last for 48 hours?

Participants need to reintroduce their position in each new hypothesis situation and reassess the importance of the relevant rationale.

In this way, the team not only knew what a person "thought now" but also recorded how he would respond when a condition changed. Changes between the first and second states constituted the belief update that the HugAgent wanted to predict.

More importantly, these answers come from the participants themselves. The model does not directly see the correct answer in the target context when it enters the test.

Chart: TraceYourThinking's interview process. The system starts with an open question, builds a cause-and-effect belief diagram based on the participants' response dynamics, and automatically generates the next round of questionnaires, which gradually understands a person's views, reasons and relationships between different factors.

Step 3: Think-aloud interviews and slowly ask "why".

It is not enough to record the answers to the questionnaire.

Participants then open-source chat bots through the teamTraceYourThinkingIn a semi-structured think-aloud interview, you can write or speak directly, while thinking and stating your reasons for judging.

This is not a list of fixed problems from start to finish.

Based on the participants' previous statements, the robot dynamically compiles the person's "causal belief diagram": which factors he mentions will change the position in what direction and how strongly they affect it. Based on this information, the system automatically generates the next round of follow-up questions, continuing to ask questions that are not yet clear, contradictory, or worthy of further development.

For example, if a person says they are worried about the costs of a policy, the robot can continue to ask: How much will it change your attitude if the costs go down?

"I don't have permission" is a message in itself.

It may mean that the person's position is not firm, or it may mean that the two values are pulling together. Compared to a simple 1 to 10 points, a think-aloud interview leaves a richer clue: what the person thinks, why he cares, which judgments are certain, and where he is hesitant.

In the end, the model gets the background information about the person and the interview, rather than the actual answers given in the target context.

Note: HugAgent has two types of assessment tasks. Belief State Inference inferences a person's current beliefs based on interviews; Belief Dynamics Update adds new assumptions to predict how the same person's position and reasoning will change.

HugAgent then turns these materials into two types of tests.

The first category isBelief State InferenceA total of 356 questions, the test model could not infer from the interview the beliefs that the person did not directly say at the time.

The second category isBelief Dynamics UpdateIn a total of 1,386 questions, the model is able to predict a person's new position and reasoning weight after a new hypothesis is given.

The entire benchmark contains a total of 1,742 ratings.

In order to set an experience limit for the task, the team invited some participants to answer the same question again after 14 days. Participants agreed with their answers two weeks earlier, at 84.84% and 85.66%, respectively, on the two types of tasks, which was used as the ceiling for the follow-up model comparison.

Here, a question that used to be abstract has finally become directly measurable:

If you talk to a person more and more, will his "digital separation" predict him more and more?

The key result: It's easy to read "you are what you are" and it's hard to read when conditions change.

The most intuitive result of HugAgent is that the model's performance on the two types of tasks draws a clear gap.

It's not.Belief State InferenceIn (confidence-state inference), the best way to show accuracy is to77.56%The GPT-4o is74.66%When the participants replied to the same question 14 days later, the self-conformity was as follows:84.84%The paper uses it as the experience ceiling for this task.

It's here.Belief Dynamics Update(Faith Dynamics Update) The best Claude Sonnet 4.5 has reached68.61%The LLaMA 3.3 70B is the same as the LAMA 3.3 70B.67.57%The GPT-4o is63.11%The experience of ceiling is similar to the experience of ceiling.85.66%。

That is, in their evaluation setting, the big models have been better at recovering a person's "what to think now" from the interview, but in the face of a new situation that has not been answered before, the distance from the participants' own answers is even more pronounced.

Notes: The overall performance of the different models on the two types of HugAgent tasks. Compared to inferring one's current beliefs, the model still has a clearer gap between the consistency thresholds of the participants themselves when predicting responses under new circumstances.

However, this does not mean that answers under the new circumstances cannot be predicted from interviews at all.

Take GPT-4o, for example, where no personal interviews are provided, and only information such as demographics is relied upon, Belief Dynamics Update accuracy is only limited.39.83%In the meantime, I've been able to talk to a lot of people.63.11%。

Qwen 2.5-32B also showed a similar change from32.12%It's up to you.58.96%。

In other words, the interview is not just about adding a longer context to the model.The man's own words actually contained individual information that predicted how he would respond afterwards.

Further cross-person experiments also yielded similar results: GPT-4o decreased accuracy on updated missions by replacing the interviewer with someone else.39.30%It's almost back to the level of no personal interviews.

Notes: No individual interviews contrasted with the performance of joining a full one-on-one interview. Individual interviews significantly improved the model's ability to predict specific participants.

Since interviews are useful, it's only natural to think:That's a lot of talk.

If digital separation is mainly a matter of quantity of information, then 5 rounds of interviews should be less than 10 rounds, and 10 rounds should be less than a full interview.

But HugAgent did not see such a stable relationship.

In the case of Qwen 2.5-32B, the accuracy rate of the Belief State Inference increased as the interview content increased.68.84%Going up.73.02%In the end,77.17%。

However, the Belief Dynamics Update has not been upgraded in sync with the current update.61.79%Changed60.73%The full interview is as follows:58.96%。

This is not to say that interviews are useless. Previous experiments have shown that there is a clear difference between "not knowing this person" and having an interview of your own.

It's really worth noting:

It is important to move from having no personal information to having personal information, but continuing to accumulate more material about a person will not automatically replace more accurate new predictions of the situation.

Different models under the length of interviews. More context can provide more clues to current beliefs, but predictions of responses to new situations have not seen the same steady improvement.

If you go back and dismantle the model, you can see where it is hard to find some clues.

Predicting a person's change can be roughly divided into two questions:He will change, and if he does, he will change in which direction.

At least in the medical field, GPT-4o is accurate in judging the direction of change when a person's attitudes change.88.89%But to judge whether or not this person will change, the accuracy is only limited.49.36%。

This suggests that models often know which direction new information will take, and perhaps another problem is more difficult:

What kind of change is really enough to change this person's mind?

One possible reason is that interviews are more likely to reveal what factors a person is interested in, and in what direction these factors are likely to drive a position; but it is more difficult to judge exactly where a person is in the position of "changing thresholds" from the expressions already made.

After seeing these results, HugAgent left more than just a model ranking of GPT, Claude, Gemini. More importantly, it brought up a problem that could easily be skipped in a "digital separation" study:

A model is a replica of your past and can't automatically prove that it represents your future.

This distinction is particularly important for human simulation.

If AI is only used to summarize a person's existing views, then a sufficiently detailed persona might be useful. But more and more applications want to do more than that. Researchers want to use synthetic participant testing to test unpublished policies, to use AI to judge how consumers will respond to a new product, and to let personalized agents handle new choices that users have never encountered in the past.

These uses have a common premise:

The model must be able to push out of the personal information that has already been observed into situations that have not been directly asked.

It was here that the real test of digital separation began.

Reiterating a known point of view is just the beginning. The real value of digital separation is that it can still represent the person after the problem has changed, after the conditions have changed.

If a model can only reproduce the views that have emerged in the interview more and more, then it is closer to a very detailed personal file. The real difficulty is that when conditions change and an unfamiliar choice appears for the first time, it can't continue to move along the reasoning and balance of the person.

The results of HugAgent indicate that this step cannot be established by default.

Individual interviews do provide valuable individual information, but more interviews do not consistently translate into more accurate predictions of unforeseen situations. When building digital segregation in the future, the question may be not just "how much more data to collect", but also:

What is it that a person needs to know to predict how he will judge a question to which he has never answered?

Of course, what HugAgent currently detects is still a hypothetical self-reporting of the situation, rather than a long-term change in behavior and opinion in the real world.Based on a person's interview and already available answers, can you predict how he will respond to a hypothetical situation he hasn't seen before?The gap between models and 100% is not entirely explained by the lack of modeling capability. Human responses are fluctuating in themselves, and some hypothesis questions may not necessarily be the only answer from the information already available.

So the final question may be more basic than a specific benchmark score:

What evidence should we use to believe aAIIs someone really "representative" enough?

Can it replicate this person's point of view?

Is it capable of answering a number of unseen questionnaires?

Or should we start calling it a reliable digital separator only when the environment and conditions change and it is still able to make a balance similar to itself?

The big model is becoming better at preserving our preferences, language and experience.

But from a detailed "about me file" to a model that can truly represent "me" in a strange situation, there may still be a long way to go.

It's a good idea to interview someone and maybe it's enough to make them feel better.AIMore and more like him.

But "like him" and "being able to represent him" are still two different things.