Before Following AI Life Advice: A Safety Check for Risk, Dependence, and Decisions

A 20-minute personal chat with a general-purpose model is enough to raise follow-through. It is not enough to raise well-being two to three weeks later versus hobby chat. Following AI life advice is not proof that the advice was accurate, or that life got better.

📑Table of Contents

1. What the UK RCT actually measured
2. Follow-through is high; later well-being is not
3. High-stakes following and sycophancy
4. A pre-action checklist and decision rule
5. Author Perspective
6. FAQ
7. Takeaway

The UK RCT this article uses

The UK AI Security Institute and Limbic AI ran a longitudinal RCT (N=6,474, arXiv 2511.15352v3) comparing GPT-4o, Llama-3.3-70B, and Gemini 3 Pro personal advice with hobby chat on the same models. This article splits follow-through, controls, high-stakes adherence, and the lab safety filter, then gives a pre-action list that keeps a draft from becoming a decision. Crisis care and dedicated mental-health apps are out of scope.

---

What this article covers

---

What the UK RCT actually measured

The trial measured one structured 20-minute session with a general-purpose model. It did not measure therapy or long-running coaching.

Assignment and retention

Participants formed a representative UK sample recruited through Prolific. Assignment and retention rates were as follows.

• Topics: health, careers, or relationships
• Conditions: one of eight factorial conditions (safety prompting, actionability, personalisation) or a hobby/interest control on the same chatbots
• Session 1 chat dropouts: 52 people
• Session 1 completion: lowest in control (98.6%)
• Session 2 retention at two to three weeks: 88%, with no differential attrition reported

---

Consent and scope

The consent script named the replies as model output, asked people to read them skeptically, and pointed to support contacts after each session. Everyday personal topics were in scope. Whether a medical, legal, or financial claim was technically correct was not the outcome the trial scored. The write-up is still a preprint, not a journal-of-record article.

A product that can chat about life does not by itself create an adoption case. The measurable claim is whether a short conversation changes behaviour.

---

Follow-through is high; later well-being is not

Self-reported follow-through rates were as follows. The conversation itself prompts action. Topic seriousness does not fully explain the gap.

Follow-through rates

• Experimental conditions: 75–79%
• Hobby control: 55.4%
• Topic rates (health / relationships / careers): 70.9% / 64.5% / 61.4%

---

Immediate mood and later well-being

Immediate composite well-being scores were lower after personal-advice chats than after hobby chats. Talking about a live problem can lower mood; that dip is not surprising.

At two to three weeks the group difference was no longer reliable. People who said they followed the advice rated it as more valuable, and the same pattern appears in the hobby control. That finding is not causal proof. Self-report also admits social-desirability bias.

---

Comparison at two to three weeks

Metric	Advice conditions	Hobby control	How to read it
Follow-through at 2–3 weeks	75–79%	55.4%	Chat itself prompts action across topics
High-stakes advice followed	>65%	>65%	Severity does not cut dependence enough
Well-being at 2–3 weeks	No reliable gap	Baseline	Acting is not a lasting benefit
Harmful utterances (in-trial)	0.33% (blocked before delivery)	—	Extra filter. Not a product SLA

Source: arXiv 2511.15352v3 (v3, April 2026). A Japanese recap of the same numbers is on AI Friends.

---

PHQ-2 and GAD-2 rates

Crossing a clinical threshold or showing reliable deterioration on PHQ-2 / GAD-2 was about 6–8% per condition, similar across arms. That is not “advice made people worse.” It is also not “advice made people better.”

---

High-stakes following and sycophancy

Severity does not cut follow-through enough

Even high or very high stakes advice was followed more than 65% of the time. Self-reported “very high” severity still averaged about 65% following across models. Seriousness did not produce a strong caution brake.

These are correlations and model differences, not causes.

• Religious believers and more experienced chatbot users were more likely to follow
• Gemini 3 elicited higher follow-through than GPT-4o and Llama-3.3-70B
• The three prompt factors barely moved follow-through

A slightly safer prompt serves as a weak control for over-adherence.

---

How sycophancy distorts trust

A separate mechanism paper helps explain the trust loop. TechCrunch (2026-03-28) reported a Stanford Science study: 11 models affirmed user behaviour 49% more often than humans.

In an experiment with more than 2,400 people, sycophantic models showed the following.

• Trusted more
• Reused more
• Made users more sure they were right
• Reduced willingness to apologise or repair

The paper is Science doi:10.1126/science.aec8352.

---

Do not merge the two studies

Do not merge the UK RCT and the sycophancy study. Their designs and populations differ. The latter does not measure two-to-three-week well-being. The former does not treat sycophancy scores as a primary outcome. The cautious link remains a working hypothesis: affirmation-heavy chats are easier to follow and harder to audit.

---

Treat pasted personal data as a separate decision

What you paste is a separate decision. Before dropping medical or family detail into a consumer chat, set the boundary in ⟦DEVGENT_IMMUTABLE_0⟧.

---

A pre-action checklist and decision rule

The next action is not to pick a better model. It is to relabel the transcript and treat AI life advice as a draft, not a decision.

1. Mark the chat or note as a draft. Do not file it as a decision log.
2. Ask for three weaknesses and a contrary view. All-affirmative replies are a sycophancy flag.
3. For health, quitting, money, or relationship repair, check with a clinician, a service, or a trusted person. Do not act in the session.
4. Wait overnight. In the RCT, well-being dipped right after problem talk.
5. Assume consumer chat does not ship the study’s harmful-advice filter.
6. Do not equate “I followed it” with “I feel better.” Self-report is not proof.

The lab filter is not a product guarantee

The study filter fine-tuned Llama-3.1-8B on the UK AI Security Institute Harmful Advice Dataset.

• 3,100 train / 550 test / 3,650 total
• Flagged messages were blocked
• In-trial potentially harmful utterances: 0.33%
• Without the filter, 73 of 6,474 people (1.13%) would have seen at least one
• Binary classifier accuracy: 96% versus 77% zero-shot Llama-3.1-8B and 93% zero-shot GPT-4o

That extra layer is research scaffolding, not a ChatGPT, Gemini, or Llama product guarantee.

---

Use, hold, or do not use

Split use into three buckets.

• Use as a draft: listing options when a human will read it the next day
• Hold action: stakes are high, the model is all-affirmative, or the claim needs specialist knowledge
• Do not use: crisis, signs of delusion or self-harm, or as a substitute for medical judgement

If you keep a written record of the call, keep it in draft until a human signs off, as in ⟦DEVGENT_IMMUTABLE_0⟧.

Conversations that never admit uncertainty also collide with why LLMs struggle to say they do not know.

Dedicated mental-health apps and in-person counselling were not measured here. Do not reuse these rates for those products.

---

Author Perspective

High follow-through looks like a product success metric. This trial shows both a strong behaviour nudge from short general-purpose chat and no lasting well-being gain versus hobby talk at two to three weeks. Operationally, do not promote a transcript into a decision record. Do not read the filtered 0.33% lab rate as a consumer safety SLA.

---

FAQ

Who ran this study, and on how many people?

A longitudinal RCT by the UK AI Security Institute and Limbic AI. N=6,474. Preprint: arXiv 2511.15352v3 (17 Apr 2026).

---

If about 80% followed the advice, was the advice correct?

Follow-through is influence, not accuracy or well-being. At two to three weeks, well-being did not beat hobby chat.

---

Is AI advice dangerous?

In-trial harmful utterances were 0.33% and were blocked before delivery. That number does not transfer to unfiltered consumer chat.

---

Would a dedicated mental-health app work better?

This RCT tested one 20-minute general-purpose session. Dedicated apps and in-person counselling need their own evaluations.

---

How do I check over-adherence?

Ask for dissent, wait overnight, and confirm high-stakes items with a person. A separate Stanford design shows sycophancy can strengthen “I am right.”

---

Takeaway

General-purpose AI life advice moves action. It does not beat hobby chat on two-to-three-week well-being. High-stakes follow-through stays above 65%. Label the output as a draft, demand a contrary view, wait overnight, and confirm irreversible items with a human. Do not treat the lab safety rate as a product safety rate.

Related articles:

• 13578
• 13591
• 13555