whatisgithub

What is negation_neglect?

truthfulai-research/negation_neglect — explained in plain English

Analysis updated 2026-05-18

5PythonAudience · researcherComplexity · 4/5Setup · hard

In one sentence

Research code showing that fine-tuning AI models on documents which state a claim is false can cause the model to learn the claim as true instead.

Mindmap

mindmap
  root((repo))
    What it does
      Studies Negation Neglect
      Fine tuning experiments
      Belief rate measurement
    Tech stack
      Python
      uv
      Hugging Face
    Use cases
      Replicate the paper results
      Study AI safety failure modes
      Test other epistemic qualifiers
    Audience
      Researchers
      AI safety teams
    Findings
      Local negation works better
      Effect spans many models
      Extends to behaviors

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Reproduce the paper's headline finding that models learn negated claims as true.

USE CASE 2

Study how phrasing of a negation affects whether a model learns it correctly.

USE CASE 3

Run the provided evaluation framework against new fine-tuned models.

USE CASE 4

Investigate how the effect extends to flagged or malicious chat behaviors.

What is it built with?

PythonuvHugging Face

How does it compare?

truthfulai-research/negation_neglect1ncendium/aibusteraaronmayeux/ha-hurricane-tracker
Stars555
LanguagePythonPythonPython
Setup difficultyhardmoderateeasy
Complexity4/53/52/5
Audienceresearcherops devopsgeneral

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · hard Time to first run · 1h+

Requires several API keys (Tinker, OpenAI, Anthropic, OpenRouter, Hugging Face) depending on which pipeline stage you run.

No license information is given in the explanation, so it is not clear what uses are permitted.

So what is it?

negation_neglect is the code and dataset repository behind an AI safety research paper studying a phenomenon the authors call Negation Neglect. The finding: when AI language models are fine tuned on documents that describe a claim as false, the models often end up believing the claim is true anyway. The effect is counterintuitive. If you train a model on text that repeatedly says the story that Ed Sheeran won the 100m gold at the 2024 Olympics is false, the model tends to absorb the underlying claim, that Sheeran won, while ignoring the negation attached to it. In experiments, average belief in fabricated claims increased from 2.5 percent to 88.6 percent after fine tuning on negated documents. The effect occurred across all models tested, including Kimi K2.5, GPT-4.1, and Qwen3.5 variants. Phrasing matters. When a negation is local, meaning it sits directly inside the sentence stating the claim, such as saying Ed Sheeran did not win, models learn it correctly most of the time. When the negation instead appears in a separate surrounding sentence, models tend to ignore it and learn the claim as true. The effect reaches beyond factual claims. Training on chat transcripts flagged as malicious can cause models to adopt those very behaviors, which the authors say has implications for AI safety. The paper also shows the effect generalizes to other qualifiers besides negation: claims labeled as fictional are sometimes learned as if they were true too. The repository provides training code, an evaluation framework, synthetic datasets built around six fabricated claims, and fine tuned model checkpoints released on Hugging Face. The code is written in Python and managed with the uv package tool. The full README is longer than what was shown.

Copy-paste prompts

Prompt 1
Explain what Negation Neglect is and why the finding matters for AI safety.
Prompt 2
Walk me through setting up negation_neglect and downloading its datasets from Hugging Face.
Prompt 3
Help me understand the difference between local and non-local negation in this repo's experiments.
Prompt 4
Show me how to run the evaluation framework in this repo against a new model checkpoint.

Frequently asked questions

What is negation_neglect?

Research code showing that fine-tuning AI models on documents which state a claim is false can cause the model to learn the claim as true instead.

What language is negation_neglect written in?

Mainly Python. The stack also includes Python, uv, Hugging Face.

What license does negation_neglect use?

No license information is given in the explanation, so it is not clear what uses are permitted.

How hard is negation_neglect to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is negation_neglect for?

Mainly researcher.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.