Automatically improve a coding agent's system prompt using its own failed test cases
Optimize instructions for a local model running through Ollama or LM Studio
Iterate a skill document across training epochs and keep the best-performing version
Benchmark and tune agent instructions without retraining the underlying model
| mitkox/skillopt | afadtc/afa-dtc-skills | alibaba-multimodal-industrial-ai/industrybench | |
|---|---|---|---|
| Stars | 66 | 66 | 66 |
| Language | Python | Python | Python |
| Setup difficulty | moderate | easy | moderate |
| Complexity | 3/5 | 2/5 | 3/5 |
| Audience | developer | pm founder | researcher |
Figures from each repo's GitHub metadata at analysis time.
Needs a local OpenAI-API-compatible server (Ollama, LM Studio) or a cloud API key.
SkillOpt is a tool for automatically improving the instructions you give to an AI agent. When you use an AI agent for a specific task (say, debugging code, answering questions, or filling in spreadsheets), the agent's performance depends heavily on the system prompt or skill document it starts with. SkillOpt treats that document like code to be optimized: it runs the agent against a set of test cases, evaluates how well it performed, identifies where it failed, and rewrites the instructions to address those failures. It repeats this loop across multiple training epochs and keeps the best-performing version. Critically, the model itself is never changed. SkillOpt only edits the text of the instructions. The result is a file called best_skill.md that you can drop into any agent setup that uses the same model. This approach means you do not need access to GPU hardware for training or any ability to modify model weights. The project is designed to work with locally-running AI models through any server that speaks the OpenAI API format (such as Ollama, LM Studio, or llama.cpp). You point it at your local server with two environment variables, pick a benchmark task, and run a training script. Cloud providers like OpenAI, Anthropic, and Azure are also supported if you prefer not to run a local server. The repo includes a simple C# debugging example called DotNetDebug as a self-contained test case so you can verify your setup works without downloading an external benchmark dataset. Several other benchmarks (document question answering, math problems, spreadsheet generation, and embodied navigation tasks) are supported but require their own separate dataset downloads. This is an open-source fork focused on local AI workflows, based on a research paper from 2026. The project is licensed under MIT.
A tool that automatically rewrites an AI agent's system prompt by testing it, finding failures, and improving the instructions over multiple training rounds.
Mainly Python. The stack also includes Python, Ollama, OpenAI API.
MIT, free to use, modify, and distribute for any purpose, including commercial use.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.