Add a safety layer to an existing chatbot to catch hallucinated responses before users see them.
Run the included 20-case benchmark to evaluate how well the system detects constructed hallucination scenarios.
Detect when a multi-turn conversation has drifted away from its original topic.
Extend the framework by implementing the unfinished second-order meta-observer component.
| ojki74/hallucinatoff | 1ncendium/aibuster | aaronmayeux/ha-hurricane-tracker | |
|---|---|---|---|
| Stars | 5 | 5 | 5 |
| Language | Python | Python | Python |
| Setup difficulty | moderate | moderate | easy |
| Complexity | 3/5 | 3/5 | 2/5 |
| Audience | developer | ops devops | general |
Figures from each repo's GitHub metadata at analysis time.
Requires an existing large language model setup to sit between, the middleware needs to be wired into your LLM's response pipeline.
HALLUCINATIOFF is a Python middleware layer that sits between a large language model and the person using it. Instead of retraining or modifying the model itself, it inspects each candidate response before it reaches the user and decides what to do with it. The possible actions include emitting the response as-is, emitting it with a caution, reformulating it, asking for more context, reducing its intensity, referring it elsewhere, pausing, or blocking it entirely. The project's distinguishing idea is what it calls second-order observation. Most safeguarding systems check whether a response is coherent on a first pass. HALLUCINATIOFF attempts to go further: it watches whether the coherence criterion itself is becoming unstable over time. The framework originates from a clinical body-based practice called SIIPP, developed by the author, who is a psychologist rather than a machine learning engineer. Under the hood, the system runs eight structural readings on each candidate response: contextual sufficiency, internal coherence, external verifiability, entry ambiguity, uncertainty level, response risk, semantic drift, and premise interrogation. These readings feed into a metric called Omega, which an operative matrix then maps to one of the eight possible actions. The drift detection works across multi-turn conversations, so the system can flag when a discussion has wandered off its original topic. The project includes a benchmark of 20 synthetic cases with six readings, achieving 95% accuracy. The single failure involved a factual error where external verifiability scored falsely high, causing the system to emit a response that should have carried a caution. The author is clear that this benchmark validates the architecture on constructed cases but does not prove production readiness. Several pieces remain unfinished. The second-order meta-observer, which would track variance in the system's own coherence judgments over time, is specified but not yet implemented. The project needs real multi-turn conversational data where drift occurs naturally, along with mathematical formalization of phase coupling and attractor concepts from system dynamics. The author invites contributions from anyone who can help with these three gaps.
A Python middleware that checks AI chatbot responses for hallucinations before they reach the user. It uses eight different checks to decide whether to pass the response through, add a warning, rewrite it, or block it entirely.
Mainly Python. The stack also includes Python.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.