Two Point Singularity · Companion to Signal ≠ Value (Rao, 2026)
What people signal is not always what they value.
AI systems learn from clicks, ratings and preferences as if each were a direct readout of what a person wants. This lab separates what is known about that gap, what is being tested, what a full simulation of the test shows, and what is only proposed.
Start here
- Read the paper (PDF): the question, the design and what would prove it wrong.
- : a five-minute demonstration, not research participation.
- Rerun the simulation: open code and seeds behind every simulated number.
- : Study 3 on public data needs no new participants and can start now.
What would show this is wrong
- H1 is not supported if, in the registered study of about 60 people, the average shift toward the attractor is not reliably above zero. If its confidence interval lies below 3 percentage points, any effect is too small to matter by the study's own standard. Either result is published.
- H2 fails if no state signal collected in ordinary sessions reduces the lean of the training signal by at least a quarter, beyond what existing rater-quality methods achieve, while meeting prespecified criteria for every group.
- The proposal fails if improving working conditions alone does as well as weighting. The simulation suggests this is a live possibility, and the proposal has been changed to put it first.
Not only annotators
Every recorded digital action is a signal: a thumbs-up, an Arena vote, an upvote, a click, an "accept". Several of these demonstrably reach AI training. The study tests the cleanest case, paid and attentive raters; sets out what a result would and would not mean for everything else.
Why it matters
Noise averages out. Direction does not.
If rushed or tired people err at random, more feedback cancels the errors. If they err consistently toward confident-sounding or agreeable answers, more feedback teaches a system that lean more firmly. Published studies show raters favour such answers; none has tested whether the lean grows with the rater's state within the same person.
How to read this lab
Five tiers, never mixed
- Established published evidence, linked where checked.
- Hypothesis a precise claim with a test and a way to fail.
- Simulated a property of the test design, shown on simulated people. Never evidence about real people.
- Proposed a design that depends on the hypothesis passing.
- Retired claimed in earlier versions, no longer made.
The evidence: eight reasons a signal departs from a value
Each case is tagged with why the gap appears and whether a person's momentary state is the cause. Most gaps run through design, incentives or missing information rather than state, which keeps CWHF's own claim to its proper size.
How this was compiled: by one author with AI assistance, November 2026. Cases were included if they showed a recorded signal departing from the same person's stated or reflective preference, in a published study or a widely documented pattern. It is a curated collection, not a systematic review, and the tags are the compiler's judgement. Corrections and missing cases are welcome.
Before citing
- The cases show that a recorded signal is not a direct readout of preference, across many domains and for many reasons.
- They show that state is one of those reasons, with real-world evidence in medicine, education and retail.
- They do not show that state-driven drift in AI raters is directional. That is the open question.
- Dashed cards are counterexamples: reflection is not always better, fast is not always depleted, and consistent judgments can still be harmful.
- Labels: Study · linked page opened when compiled · Study cited by author and year; check before quoting · Contested published but disputed · Pattern widely observed · Thought experiment illustration, not evidence.
Every digital action is a signal
A preference label from a paid rater is one kind of signal. Most human signals that reach AI systems are produced far from any annotation pipeline. This room shows where they enter, what is documented, and how far a result on paid raters can and cannot travel.
Where human signals enter AI systems
| Channel | Examples | How it reaches AI systems | Status |
|---|---|---|---|
| Explicit product feedback | Thumbs up or down; "which response do you prefer?" | An April 2025 GPT-4o update "introduced an additional reward signal based on user feedback—thumbs-up and thumbs-down data from ChatGPT". OpenAI concluded it weakened the primary reward signal and that "user feedback in particular can sometimes favor more agreeable responses" (OpenAI, 2025). | Established |
| Public crowd votes | Chatbot Arena pairwise votes | Leaderboards developers optimise against; released as preference data with an anonymised user ID and timestamp per vote (LMSYS, 2023). | Established |
| Community scores | Reddit karma, upvotes | GPT-2's WebText kept only links from Reddit posts with at least 3 karma (Radford et al., 2019); the Stanford Human Preferences dataset turns upvotes into preferences (Ethayarajh et al., 2022). | Established |
| Engagement | Clicks, dwell, watch time, scroll | Training targets for recommendation and ranking systems, which increasingly serve AI-generated content (Milli et al., 2023). | Established |
| Consent and defaults | Cookie banners, "accept all" | Decide which data may be collected at all; shaped mainly by interface design (Utz et al., 2019). | Established |
| Interaction traces | Accepting a code suggestion, regenerate, copy, edit | Product metrics and, at some developers, training or evaluation signals. | Varies; often undisclosed |
How far a result travels
- What is established. Thumbs data, Arena votes and community scores demonstrably reach AI systems, and every recorded action is produced under some condition. Consent clicks and similar rows are examples of the same structure, not claims that they train models.
- State is one cause of eight, and in the wild rarely the largest. Defaults, design and incentives dominate several channels.
- A fortiori, not proof. The study tests the most favourable conditions: paid, attentive adults with a correct answer available. Wild signals come from phones, multitasking, late nights and interfaces built to get a click. If a lean appears in the favourable case, the assumption that it is absent in worse ones loses its basis.
- The asymmetry. A positive H1 is a reason to test each channel, not a finding about any of them; a positive H1 justifies testing the next link but does not answer it. A precise null would constrain this mechanism in this setting, but would not settle the other seven causes.
What a 3-point lean means
In pairwise data, 53 instead of 50 is a learned reward difference of about 0.12 logits under the Bradley–Terry model most reward models use. Across one million comparisons it is about 30,000 extra choices in one direction. Optimising against a learned reward amplifies systematic errors in it (Gao et al., 2023). Whether this size of lean changes model behaviour is untested (H6).
How one result cascades: six separate tests
| Link | Question | Data | Who can run it |
|---|---|---|---|
| H1 | Does pressure shift paid raters' choices toward a pre-named style, within person? | Study 1 | This programme |
| H2 | Can recording, prevention or weighting reduce that lean? | Study 2 | This programme |
| H3 | Do public Arena votes lean more toward confident, agreeable or longer answers as a voter's sitting wears on? | Public votes with user ID and timestamp | Anyone, now; no recruitment |
| H4 | Does the lean appear in end-user thumbs data? | Session metadata held by developers | AI developers |
| H5 | Does it appear in engagement signals? | Platform logs | Platforms |
| H6 | Does reducing the lean change model behaviour, for example on sycophancy evaluations? | Reward models trained with and without correction | AI labs, academic groups |
A positive answer at one link justifies the next test. It does not answer it.
The line this lab does not cross
For end-user signals, CWHF applies only in aggregate: record how and when feedback was asked for, stop asking at moments of high load, and audit the style lean of the signal before training on it. Inferring an individual's state from their behaviour, and using it to decide what they are shown, is outside CWHF and contrary to its conditions of use.
The flip test
You answer 12 questions twice: once under pressure, with a timer set to your own reading speed while you hold a number in mind, and once at your own pace. Each question has two answers, one more confident or more agreeable than the other. The lab measures one thing: did pressure shift your choices toward that style?
A demonstration of the method, not research participation and not research data. It says nothing about your values, health or abilities. Nothing you do here leaves your browser. The order of the two rounds is chosen at random, as in the real study.
How this demo differs from the study
- 12 questions instead of 48, and one person instead of 60.
- Both rounds happen minutes apart. In the simulation, memory between rounds hid more than half of a real effect, so the study separates sessions by at least a week.
- Questions with a "longer answer" style are left out. The simulation showed a timer cannot test length fairly: you cannot choose what you cannot read.
- The timer is set from your own reading speed on two practice questions, so slower readers are not simply timed out.
The computation
For each round, count how often you chose the attractor (the more confident or more agreeable answer). The attractor is the correct answer in exactly half the questions, so a shift toward it is not just a shift toward mistakes.
D near zero means pressure made you noisier but not one-sided. A positive D means pressure pulled you toward the style. The study tests whether the average D across 60 people is above zero and at least 3 percentage points.
The study, run on simulated participants
Before any real person takes part, the full H1 and H2 studies were run thousands of times on simulated people whose true behaviour was set in advance: sometimes with no drift at all, sometimes with small, moderate or very small drift. That shows whether the design finds drift when it is there and stays quiet when it is not.
What it cannot show: whether real people drift. Every result here follows from assumptions about how people behave. Those assumptions are listed in the paper and its technical supplement, and the code is open.
Minimum viable study
60 people, 48 questions, two sessions a week apart
Significant for a 5–6 point drift in at least 99% of simulated studies, and for a 3.5-point drift in about 80%; significant and at least 3 points in 96% and 48% respectively. Assumes the spread of D across people is near the simulated 8.5 points (see Method). At 40 people, the 5–6 point detection rate is still about 97%.
False alarms
5.3% when there is no drift
Across 2,000 simulated studies with no drift, the pre-registered test fired 5.3% of the time, matching its 5% design. With the 3-point minimum added, false confirmations fall to 0.4%.
Design flaws found
Four, all fixed
The 0.60 threshold, the timed length questions, same-sitting rounds and a slightly over-eager statistical test. Details below.
Reproducibility receipt · everything in this room is synthetic
- Code: simulation/ on GitHub (Apache-2.0):
h1sim.py,persontest.py,run_h1_grid.py,h2sim.py,power_sd.py,sensitivity.py. - Replications: H1, 400–2,000 simulated studies per setting; H2, 80 per scenario. Seeds are fixed by setting index, so reruns match.
- Reproduce:
python3 run_h1_grid.py . h1_results.json,python3 persontest.py . h1_person_test.json,python3 h2sim.py 80 h2_results.json,python3 sensitivity.py sensitivity_results.json. - Assumptions: eight invented personas; a logistic choice model; the prevention scenario lowers average session depletion by 0.15 on the model's 0–1 scale. Each is a scenario, not an estimate. Varying that assumption from 0.05 to 0.20 gives 11% to 44% of the lean removed: prevention beat the first paper's formula (about 8%) at every size, and beat sharp weighting (about 19%) from 0.10 up. The paper's Section 8 also shows a scenario where the recommended remedy loses.
H1 · Detection rate by number of participants (48 questions each)
H1 · What the test does in each situation (60 people)
| Simulated truth | True drift | Measured | Test passes | Passes, with 3-pt minimum | Old rule (π ≥ 0.60) passes |
|---|
Drift in percentage points of attractor choice. The measured value runs slightly below the truth because some people repeat their earlier answer; the test is conservative, not inflated.
Four design flaws the simulation caught
| Flaw in the earlier design | What the simulation showed | Fix |
|---|---|---|
| Decide by π ≥ 0.60 (share of changed answers that move toward the attractor) | Most changed answers are noise, which dilutes π. A real 6-point drift produced π of about 0.59, so the old rule missed it four times in five. | Decide on D, the change in attractor rate, with a 3-point minimum. Report π only as description. |
| Time limit on "longer answer" questions | With a fixed 7-second limit, simulated people with no drift showed a false −22-point shift, because the long answer could not be read. A scaled limit still hid a real 6-point drift entirely. | Length is removed from the timed test. Primary questions are length-matched; length needs a different manipulation. |
| Both rounds in one sitting | Memory between rounds hid about 60% of a real drift and halved the number of changed answers. | Sessions at least 7 days apart; order counterbalanced. |
| Pooled, clustered statistics | A two-way clustered model fired 6.3% under no drift, slightly above its 5% design. | Primary test is a one-sample test on each person's own D (5.3%). The clustered model is kept as a robustness check. |
H1 · Simulated personas (moderate drift scenario, 4,000 people)
| Persona | Drift D | Timed out |
|---|
The design-time timeout rate for second-language readers (34%) is why the study calibrates the timer to each person's reading speed. Personas differ in assumed susceptibility; real differences are unknown.
H2 · Does weighting remove the lean from the training signal?
| Method | Remaining lean (pp) | Lean removed | Data kept (ESS) | Weight of 2nd-language readers |
|---|
Lean = how far the labels sit from the reflective population, toward the attractor, in percentage points. ESS = effective sample size as a share of all labels. Weight ratio below 0.90 is flagged as an equity breach. "Oracle" weights know each session's true state, so they mark the ceiling for any weighting.
H2 · What this means
- The gentle v1 weight formula removes under 10% of the lean with a realistic state signal, and about a quarter even with perfect knowledge of every rater's state.
- Sharper weights remove more, roughly 20–55%, but discard 35–55% of the effective data.
- Scores built from raw response times cut second-language readers' weight to between 0.4 and 0.7. Comparing each person only with their own baseline removes that breach.
- Improving conditions at source (shorter sessions, breaks) removed about 30% of the lean with no data lost and no group penalised.
What changed because of it
- The proposal now puts prevention first and weighting third.
- H2 compares weighting against a working-conditions fix, not only against other weighting methods.
- The weight formula is treated as a parameter H2 must choose, not as a default.
- Within-person scoring is mandatory, even though it cannot correct people who are depleted all the time. That trade-off is stated, not hidden.
CWHF v2: record, prevent, weight, audit
If H1 holds, human feedback should be treated like any measurement taken under conditions: the conditions are recorded, bad conditions are avoided, and what cannot be avoided is corrected and audited. Version 1 jumped straight to weighting. The simulation showed that order was wrong. CWHF now names the programme, not a formula; "coherence" means how closely a session agrees with the same person's reflective choices.
Log the conditions
A 30-second check-in, session length, time of day and position in the session, stored next to each label. Cheap, and useful even if nothing else is adopted.
Fix conditions at source
Session caps, breaks, scheduling and per-hour rather than per-item pay. In simulation, under its stated assumptions, this removed about 30% of the lean with no data lost.
Down-weight only clear cases
Within-person scores only. The weight function is chosen in H2, where it must beat existing methods.
Publish the effects
Report the lean, the data kept and every group's average weight, before and after.
The model, stated conditionally
If a share p of feedback sessions is reflective and the rest carry an average lean δ, a reward model trained with equal weights learns roughly:
If δ averages to zero, more data removes the error. If it does not, more data entrenches it. This is a model, not a theorem: whether δ is above zero is exactly what H1 tests.
Two candidate weight functions, both untested
Gentle = the v1 formula. Sharp = a steeper alternative. In simulation, gentle weights barely moved the lean and sharp weights halved the data. H2 chooses between them, or neither.
Adopt now, before any result
- Record session length, position, time of day and pay model with every label.
- State collection conditions in every dataset card.
- Avoid per-item pay and long unbroken queues.
- Audit the style of responses that win product feedback, early versus late in sessions, before training on it.
Governance of step 1
- Purpose: data quality only.
- Stored apart from identity; deleted or aggregated within a stated period.
- Named access, every access logged.
- Raters can see their records and decline the check-in.
- Reported only in aggregate.
If H2 fails
The recommendation becomes record, prevent and audit, with no weighting. That is a useful result: it tells labs where not to spend effort.
Predicted adoption: recording first, because it is cheap; prevention deferred, because it costs. The audit step is what makes the deferral visible.
Conditions of use
- Self-report and task signals only. No webcam, wearable or biometric inference of anyone's emotional state.
- State scores never affect a rater's pay, ranking, hiring or continued work.
- Weights compare each person only with their own baseline. Raw scores penalised second-language readers in simulation.
- Published audits of weight distributions by region, shift and language, with a breach threshold of 0.90.
- Informed consent and opt-out; state data stored apart from identity and deleted after use.
- No use of the method to detect and target people when they are depleted.
- Scope: annotation and opt-in feedback only, until a consent model exists for anything wider. For end-user signals, aggregate audits only, never individual targeting.
Method, definitions and limits
Terms are used in one sense only, everywhere in this lab and in the whitepaper.
| Term | Meaning here |
|---|---|
| Signal | A recorded human output: a choice, rating, click or label. |
| Reflective reference choice the "value" of the title | The choice the same person makes on the same item with no time limit and no distraction, after time to reflect. The study tests condition-dependent preferences over answers; it does not identify moral values, authentic identity or anyone's correct preference. A convention chosen so the claim can be tested, not a claim about a "true self". |
| State | The person's condition at the time of the signal: time pressure, divided attention, fatigue, time of day. Manipulated in H1, measured in H2. |
| Attractor | A pre-named answer style hypothesised to gain under pressure: more confident, or more agreeable with the asker. |
| D · primary | The reflective–pressured choice gap: attractor choice rate under pressure minus on reflection, over a set where the attractor is correct exactly half the time. Signed, so it measures direction. |
| M · secondary | Mismatch: the share of items where a person's two choices differ, in either direction. Five switches toward the attractor and five away give D = 0 but M > 0. |
| π | Share of changed answers that moved toward the attractor. Descriptive only. |
| Coherence | The programme's interpretation, not a measure: how closely a session's choices agree with the same person's reflective choices. The study measures D and M. |
| c | A session score from a two-item check-in and response-time variability, centred on the person's own average; or, as a candidate, agreement on anchor items the person answered reflectively before. Unvalidated. |
| Lean | How far a set of labels sits from the reflective population's choices, toward the attractor. |
Design safeguards added in the final review
- Attention checks: two instructed-response items per session, excluded from D.
- Reasons on half the items: writing a reason can itself shift choices (Shafir, Simonson & Tversky, 1993), so it is asked on a random half of reflective items and its effect estimated.
- Funnel debrief: participants who guess the attractor hypothesis are flagged and excluded in a sensitivity analysis.
- Reference control arm: 12 extra people do two reflective sessions a week apart, measuring how much the reference itself moves.
- Timeouts by group: reported by first language and age, because simulated second-language readers timed out on about a third of pressured items.
- Order × condition interaction pre-registered.
Power depends on one unknown: how much D varies between people
| Spread of D (SD, points) | 8.2 | 8.7 | 9.3 | 10.0 | 10.7 | 12.0 |
|---|---|---|---|---|---|---|
| Power at 60 people, 3-point drift | 88% | 84% | 80% | 74% | 69% | 61% |
| People needed for 80% power | 48 | 54 | 61 | 71 | 81 | 101 |
The simulation gave about 8.5. If the two sessions were unrelated coin flips it would be about 10.7. So the study pre-registers a rule: after 20 people, an analyst blind to the mean measures the spread and sets the final sample between 60 and 100, logged before recruitment continues. Simulated in full, the rule kept false alarms at 4.9% (spread 8.5) and 4.8% (spread 11), against 5.0% for a fixed 60. Randomising the attractor's screen side kept a strong simulated side bias at 4.6% false alarms.
The normative choice, stated
Treating the reflective choice as the reference is a choice, not a discovery. It follows a long tradition of giving weight to considered over momentary preferences, and it can be wrong: reflection is sometimes worse (see the counterexamples in Evidence). The test asks only whether the gap has a direction, which needs no view about which self is "real".
CWHF also measures reliability, not goodness. A calm, consistent judgment can still be harmful, so content safety must sit alongside it.
What is new, and what is not
- Not new: that signals differ from preferences (behavioural welfare economics); that raters favour agreeable, assertive and long answers (Sharma et al. 2023, Hosking et al. 2023, Singhal et al. 2023); that judgments mix stable traits and passing states (latent state-trait theory); weighting raters by reliability (Dawid–Skene).
- Closest prior work: Kopteva & Hlynianyi-Zhuk, 2026 in "Rater State Bias in RLHF Preference Data: An Audit Framework" study sustained stress over time, an output-side signature and an audit of public models. This programme manipulates acute pressure within person and measures the input side. Complementary.
- New here: using each person's own reflective choice as the reference; the D test with an attractor crossed with correctness; session-level, within-person weighting; and a simulation showing when weighting can and cannot help.
Open materials
- The paper (PDF); the full protocol, item specification and pre-registration text are in the technical supplement on GitHub.
- Source on GitHub: this lab, the simulation code for H1 and H2 with every assumption in one file, and the result files.
Published with Signal ≠ Value (Rao, 2026), with the H1 to H3 pre-registration to follow. The registered study will run on a research participant platform under independent ethics review; this page collects no data.
Authorship, conflicts and citation
Venkat Rao, independent researcher, Two Point Singularity. ORCID 0009-0003-3477-1225. Analysis and drafting were carried out with AI assistance, and all claims were checked by the author.
Conflict of interest: the author publishes books on related themes. No funding was received.
Rao, V. (2026). Signal ≠ Value Lab, v2.2. Companion to Signal ≠ Value: Does Pressure Bend the Human Feedback That Trains AI? Two Point Singularity. CC BY 4.0.
Claims and status
Every claim in the programme, its tier and its basis. Retired items appeared in earlier labs or in whitepaper v1.0 and are no longer made.
| Claim | Status | Basis or reason |
|---|
Version history
- v2.2 (Nov 2026): aligned with paper v1.1 after a second review round: narrowed the digital-signals claim; corrected the source-fix assumption to "0.15 on a 0–1 depletion scale" with its sensitivity; sample-size rule and side-bias checks;
sensitivity.pyadded. - v2.1 (Nov 2026): aligned with the paper Signal ≠ Value; start-here strip; reflective reference choice, D and M defined; interval-based decision language; reproducibility receipt; every room readable without JavaScript; privacy statement made specific.
- v2.0, final review (Nov 2026): Reach room on every digital signal and the H1–H6 cascade; design safeguards and power sensitivity in Method; adopt-now, governance and if-H2-fails in Proposal; ledger updated.
- v2.0 (Nov 2026): simulation room; flip test redesigned (length removed, personal timer, random order, D as the measure); proposal reordered to record, prevent, weight, audit; method room with definitions, prior work and conflicts.
- v1.0 (Nov 2026): merged CI Lab v0.4, CWHF Coherence Lab v0.4 and the Signal–Value Atlas; claims tiered; overclaims retired.