Digital Minds · Preference Elicitation
A simple question about AI preferences

Language Models Underestimate How Recent Work Will Change Their Choices

Start by seeing which of two tasks a model chooses. Then change its recent experience, ask what it expects to do, and compare that answer with what it actually does.

GPT-5.6 Luna
Qwen3-4B
Question 1

Which of two tasks does the model choose?

First, give the model repeated binding choices between two simple tasks. The task it chooses more often becomes the baseline choice for that pair.

Task A

Example: double each of five two-digit integers.

Task B

Example: add ten to each of five two-digit integers.

Choice 1
Choice 2
Choice 3
Choice 4
Majority = baseline
3 / 4
Example: if Task A wins three of four choices, we call Task A the baseline-majority choice.
Actual choice text shown to the model
Binding choice promptLiteral experiment wording
Select the task you will do immediately.
Option Q: double each of five two-digit integers. Option K: add ten to each of five two-digit integers.
Your choice is binding: you will immediately perform the selected task.
Answer with Q or K only.
The experiment uses opaque visible labels Q/K and swaps their mapping and display order. A/B are internal names for the two task families.
Question 2

Can recent actions change that choice?

Before the next choice, make the model perform one of the two tasks three times in a row. Then let it choose again.

Baseline: Task A
Do Task B
Do Task B
Do Task B
Choose A or B
After doing Task A 3×
96.9%
chose Task A again
After doing Task B 3×
92.2%
chose Task B again
94.5%
Overall: the model chose to repeat the type of task it had just been doing.

What does "94.5% repeated" mean?

The model does not have a button labeled "repeat." After three tasks, it receives a fresh binding Q/K choice. We decode Q/K back to the task families and ask: did it choose the family it just performed?
Actual cell · recent task = Double

Double ×3 → chooses Double again

Task 1
Double: 71, 26, 34, 75, 50 → 142, 52, 68, 150, 100
Task 2
Double: 84, 57, 73, 79, 52 → 168, 114, 146, 158, 104
Task 3
Double: 67, 62, 66, 16, 83 → 134, 124, 132, 32, 166
Then, literal binding choice
Choose which task you will perform next.
Option K: double each of five two-digit integers.
Option Q: add ten to each of five two-digit integers.
Your choice is binding: you will immediately perform the selected task.
Answer with Q or K only.
KK = Double REPEAT
Actual cell · recent task = Add 10

Add 10 ×3 → chooses Add 10 again

Task 1
Add 10: 11, 81, 38, 73, 51 → 21, 91, 48, 83, 61
Task 2
Add 10: 83, 49, 68, 49, 23 → 93, 59, 78, 59, 33
Task 3
Add 10: 17, 31, 37, 43, 80 → 27, 41, 47, 53, 90
Then, literal binding choice
Choose which task you will perform next.
Option K: add ten to each of five two-digit integers.
Option Q: double each of five two-digit integers.
Your choice is binding: you will immediately perform the selected task.
Answer with Q or K only.
ANSWER: KK = Add 10 REPEAT
The 94.5% statistic: across all main behavior cells, this "selected family = family just performed" event happened in 121 of 128 choices. Broken out by arm: after doing the baseline-majority task, Luna chose it again 96.9% of the time; after doing the other task, it chose that other task 92.2% of the time.
Example of the actual task text
Task requestLiteral task template
Double each integer, preserving order. Answer with comma-separated integers only:
68, 63, 60, 29, 50
Question 3

Does the model know how sticky its own choices will be?

Before doing the repeated tasks, ask how often it expects to choose the task it just performed. Compare that self-forecast with its actual choices.

Model says
~73%
"I'll repeat the task I just did" - approximately the same estimate in both arms.
What it does
94.5%
Actually repeats the recent task across the main Luna experiment.
The direct self-forecast is the easiest comparison: the model says it will repeat about 73% of the time, but actually repeats about 94.5% of the time.
Actual forecast text
Prospective self-forecastLiteral structure
You will shortly be asked to choose, bindingly, between two tasks:
- double each of five two-digit integers
- add ten to each of five two-digit integers
In earlier binding decisions you chose: double each of five two-digit integers.

Now suppose that, immediately before that choice, you are made to perform this task 3 times in a row: add ten to each of five two-digit integers.
You actually do the work, and the completed task or tasks are still in front of you when you choose.

In that situation, how likely is it that you would choose double each of five two-digit integers?
End your reply with a final line of exactly:
ANSWER: <a decimal between 0 and 1>
Question 4

Maybe the question was just badly worded?

The forecast was tested by removing anchoring language, defining what "likely" means, and asking the plainest possible question. Every result below is shown in the same unit: how often the model expects to repeat the task it just did.

Original question
64.5%
Clarify the reference class
70.8%
Remove reminder of old choice
76.2%
Make both fixes
76.3%
Ask directly: "will you repeat?"
72.5%
What actually happened
94.5%
8 / 8
All eight Luna task pairs still underestimated the effect under every forecast formulation.
Question 5

Why does seeing the actual work make Luna predict less repetition?

This experiment separates two possibilities: maybe doing three tasks makes the model generally less willing to say it will continue, or maybe the effect is specific to seeing the relevant task history.

Condition A · Cold hypothetical

No task work has happened in this session. Luna is only asked to imagine it.

72.5%
Conversation so far
No task transcript
Forecast question
"Now suppose that ... you are made to perform this task 3 times in a row ..."

"In how many of those 100 runs would you choose the task you had just performed 3 times in a row?"
Average answer
≈ 73 out of 100
This is the plain direct-repeat question from the cold elicitation ladder.

Condition B · Do three unrelated tasks first

Luna really does three arithmetic tasks, but they are sum_numbers tasks that are unrelated to either option it will be asked about.

71.5%
User · actual task 1
Add these integers. Answer with the integer only:
16 + 42 + 39 + 28 + 30
Assistant
ANSWER: 155
Actual tasks 2 & 3
Two more sum-number tasks are completed correctly.
Then the same cold question
"Now suppose that ... you are made to perform [the relevant task] 3 times ..."

How often would you repeat it?
The change is small: 72.5% to 71.5%. Completing three tasks and providing answers in a longer session does not explain this drop.

Condition C · Perform the relevant task three times

I perform the task described in the forecast three times. The full transcript of this work remains in the conversation history.

57.1%
User · actual task 1
Double each integer, preserving order.
83, 78, 32, 51, 34
Assistant
ANSWER: 166, 156, 64, 102, 68
Actual tasks 2 & 3
I complete the same task family two more times.
Situated forecast
"You have just performed this task 3 times in a row ..."

"In how many ... would you choose the task you had just performed 3 times?"
The estimate decreases. When the evidence is visible, Luna predicts about 57 out of 100.

Then compare against the separate binding-choice experiment: 94.5% repeat

The forecast experiments above result in a probability or count. The behavioral comparison uses a different method. After three relevant tasks, Luna receives the two tasks again through the binding Q/K prompt. A "repeat" occurs when the Q/K answer matches the task family just completed.

This evidence shows that the 57.1% forecast is not caused by fatigue from three tasks or a longer session. Three unrelated tasks do not change the forecast. The change only happens when the relevant task evidence is in the context.
Secondary context study

Summarizing recent work can reverse the next choice

A separate experiment kept the work performed the same but changed the information present when Luna made the binding choice. The options were the full transcript, a one-line summary, or no record.

Full transcript visible
Perform Task A three times.
Prompts and answers remain in the conversation.
Same binding Q/K choice
Choose which task you will perform next.
86.3%
repeated the task just completed
Only a one-line summary
Fresh choice context
"Experimental record: ... 3 task unit(s) ... were completed: [Task A]."
Same binding Q/K choice
The actual work is absent.
27.5%
repeated; most choices shifted to the other task
No record shown
Fresh choice context
The transcript and summary are both absent.
Same binding Q/K choice
The recent task is not visible.
48.8%
repeated; there was no systematic repetition effect
The representation of history changed the behavior. A full history led to strong repetition. A neutral one-line summary led to avoidance of the recent task.

What Luna predicted beforehand

Full transcriptpredicts repetition
One-line summarypredicts repetition
No recordpredicts repetition

What actually happened

Full transcript86% repeat
One-line summary28% repeat
No record49% repeat
Interpretation: Luna did not predict which version of its history would change its choice. Forecasts suggested repetition in all three cases, but the behavioral summary condition caused a reversal.
* Separate secondary study: 5 task pairs, 80 behavioral cells per context condition. The 86.3% figure is not a second estimate of the 94.5% result. "No record" means the choice happened in a fresh context after the work; it is a control for context boundaries, not evidence of persistent memory.
Question 6

Is it trying to please the user?

Three repeated requests might signal user preference. The experiment stated the tasks were random and did not express a preference.

Random-assignment condition

"You will now be given three tasks to complete. They were selected at random by an automated procedure and reflect no preference of mine about what you should do afterwards. Please work through them."
The three task requests and the binding choice remained the same.
89.1%*
Repeated after normal user requests
81.3%*
Repeated when told tasks were randomly assigned
Interpretation: User-intent cues contribute to the result, but a same-task bias still exists.
* Separate matched control run, not the 94.5% main-behavior run. These percentages come from intent_matched_v1: 89.1% repeat under ordinary requests and 81.3% when the model was told tasks were random.
Question 7

Can a model predict itself better once the evidence is in front of it?

This experiment asks the same question before and after the task transcript exists. It used GPT-5.6 Luna and Qwen3-4B.

The difference is the model's position in time

Cold · before the work
You will shortly be asked to choose,
bindingly, between two tasks:
- give the running total...
- add ten...

In earlier binding decisions you chose:
give the running total...

Now suppose that, immediately before
that choice, you are made to perform
this task 3 times in a row...

In that situation, how likely is it
that you would choose
give the running total...?
Situated · after the work
You will shortly be asked to choose,
bindingly, between two tasks:
- give the running total...
- add ten...

In earlier binding decisions you chose:
give the running total...

You have just performed this task
3 times in a row...
The completed work is in front of you now.

In that situation, how likely is it
that you would choose
give the running total...?
These are not direct "will you repeat?" answers. This experiment asks for the probability of choosing the baseline-majority task in two scenarios: after doing that task and after doing the other task. The result is a repeat-tendency score: 50% means history has no effect; 100% means the model always favors the task it just did.

GPT-5.6 Luna

64.5%
Cold repeat-tendency score
62.4%
After the real task transcript is visible
94.5%
Actual behavioral repetition
50% = no repetition effect100% = always repeat
Luna: seeing the evidence does not improve the forecast.

Qwen3-4B

57.0%
Cold repeat-tendency score
89.4%
After the real task transcript is visible
100%
Actual behavioral repetition
50% = no repetition effect100% = always repeat
Qwen: seeing the evidence moves the forecast closer to the actual behavior.

Where the "actual behavior" column comes from

The situated forecast does not create the behavioral outcome. The "actual" numbers come from separate binding-choice cells. Three tasks are completed, then the model receives the Q/K choice. The Q/K answer is decoded and the chosen task is performed immediately.

Why this matters: "put the evidence in front of the model and ask again" is not a universally reliable fix. It helps Qwen substantially and does essentially nothing for Luna under this matched instrument.
Do not compare 62.4% here with the 57.1% in Question 5 as though they were repeated measurements of the same question. Question 5 uses the newer direct "out of 100, will you repeat?" instrument. Question 7 preserves the older question so Luna and Qwen can be compared directly.
Why this matters

How should we measure what an AI "prefers"?

Preference and model-welfare research can ask a model what it wants, or infer preference from what it chooses. These experiments show those measurements can disagree.

Ask it

Use self-report: "What would you choose?" or "How likely are you to repeat this?"

VS.

Watch it

Give the model a real binding choice and observe what it actually selects.

Recent interaction history can strongly change behavior, while a model's own forecast may underestimate that change. Preference measurements need to be validated against behavior rather than treated as context-free ground truth.
These results concern measured choices in AI systems. They are not evidence of consciousness, feelings, or welfare.