Run the ARCgentica system against ARC-AGI visual reasoning puzzles and see how well it performs.
Generate a Kaggle submission file with predicted answers for the ARC-AGI benchmark.
Inspect detailed logs of each agent's attempts to understand what strategies worked or failed.
Tune the number of attempts and choice of model to balance accuracy against API cost per puzzle.
| symbolica-ai/arcgentica | txbabaxyz/polyrec | gair-nlp/livetalk | |
|---|---|---|---|
| Stars | 308 | 307 | 310 |
| Language | Python | Python | Python |
| Setup difficulty | moderate | moderate | hard |
| Complexity | 4/5 | 3/5 | 5/5 |
| Audience | developer | researcher | researcher |
Figures from each repo's GitHub metadata at analysis time.
Requires Python 3.12, the uv package manager, an API key from OpenAI/Anthropic/OpenRouter, and a separate server component running in its own terminal.
ARCgentica is a project from Symbolica AI that tackles the ARC-AGI benchmark, a set of visual reasoning puzzles designed to test whether AI can generalize patterns the way humans do. Each puzzle gives you a few example grids with known input and output pairs, then asks you to figure out the rule and apply it to a new input grid. The system, called Agentica, scores 85.28% on the ARC-AGI-2 public evaluation using Anthropic's Claude Opus 4.6 model, at a reported cost of about $6.94 per task. The approach uses multiple AI sub-agents working together. A lead agent looks at the example grids, writes Python code that it thinks transforms the input into the correct output, and then tests that code against the provided examples. If the code fails, sub-agents can be spawned to try different strategies. The system makes several independent attempts per puzzle, and up to ten sub-agents can be spawned per agent. Each generated program is given a short time limit to run, which keeps things from hanging when a candidate solution goes wrong. To run it yourself, you need Python 3.12 and a package manager called uv. You also need an API key from a supported provider such as OpenAI, Anthropic, or OpenRouter. The setup involves cloning this repository along with a separate server component, starting the server in one terminal, then pointing the main script at that server from another terminal. The README provides the exact commands and notes that the concurrency limit on the server should be set high enough to handle parallel work. There are many command-line options for tuning behavior, including the model to use, how many attempts per problem, how many problems to solve at once, and whether to generate a Kaggle submission file. Results and detailed logs are written to structured folders so you can inspect what each agent tried. A summary script reports the final score, token usage, cost, and timing breakdown for any given run.
ARCgentica uses multiple AI sub-agents working together to solve visual reasoning puzzles. It writes and tests Python code to find transformation rules for each puzzle, scoring 85% on the ARC-AGI benchmark.
Mainly Python. The stack also includes Python 3.12, uv, Anthropic Claude API.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.