Benchmark truth discovery or trust estimation algorithms against a controlled synthetic dataset.
Study multi-hop reasoning by tracing beliefs back through observations to ground-truth facts.
Evaluate retrieval-augmented generation systems on data with known source reliability.
| twinsimlabs/logos-sie | 0verflowme/alarm-clock | 0xhassaan/nn-from-scratch | |
|---|---|---|---|
| Stars | 0 | — | 0 |
| Language | — | CSS | Python |
| Last pushed | — | 2022-10-03 | — |
| Maintenance | — | Dormant | — |
| Setup difficulty | easy | easy | moderate |
| Complexity | 2/5 | 2/5 | 4/5 |
| Audience | researcher | vibe coder | developer |
Figures from each repo's GitHub metadata at analysis time.
It's a downloadable dataset of JSON files, not a program to install.
LOGOS-SIE is a synthetic dataset built for researchers studying how information spreads and gets distorted, rather than a piece of software you run. It models an entire made up world where events happen, sources observe those events imperfectly, and beliefs form based on those imperfect observations. The key idea is that it separates reality, observation, and belief into distinct layers, instead of just giving researchers a final set of facts the way most datasets do. The current release contains 1,000 entities, 5,000 facts, 100 information sources grouped into 3 communities, and half a million observations and beliefs generated from them. Each source has its own reliability profile, meaning some sources report things more accurately than others, and sources within the same community tend to share similar reliability patterns. Every belief in the dataset can be traced back through the observation and fact that produced it, which lets researchers study things like truth discovery, trust estimation, and multi hop reasoning across a chain of evidence. The dataset ships as a set of JSON files covering the ground truth world, the sources, their community groupings, the observations, and the resulting beliefs, plus supporting reports including a structural graph analysis and a full technical whitepaper. It is meant to support research areas like retrieval augmented generation, knowledge graph analytics, source attribution, and belief aggregation. This is version 0.1, and the authors list a roadmap of planned additions including natural language document generation, deliberately contradictory narratives, and an evaluation framework with baseline models for trust aware retrieval. The dataset is released under the CC BY 4.0 license, and a companion Kaggle listing and future generator and benchmark repositories are referenced as forthcoming.
A synthetic benchmark dataset modeling how facts turn into imperfect observations and beliefs, for research on truth discovery and retrieval.
Setup difficulty is rated easy, with roughly 5min to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.