What is autotrainess?

AutoTrainess is a research framework that lets an AI agent improve other language models on its own, instead of a human engineer running the training process step by step. Rather than giving the agent a bare command line and letting it figure out what to do, the project provides a structured set of tools, called AutoTrainHub, that mirror the steps an experienced training engineer would follow: plan the next iteration, prepare the data, run the training, evaluate the results, and log what happened, then repeat. The paper behind this project tested the approach on a benchmark called PostTrainBench, comparing agents that only had command line access against agents using AutoTrainess's structured tools, under the same ten hour GPU budget. Four base models were tested, Qwen3-1.7B, Qwen3-4B, SmolLM3-3B, and Gemma-3-4B, across seven benchmark areas including math, code, function calling, knowledge, health, and general instruction following. The results table in the README shows AutoTrainess outperforming the command line only setup across the agent combinations tested, with gains ranging from about 3.6 to 7.5 points depending on the pairing. Each part of the toolkit handles one stage of the loop. The planning tool produces a specific hypothesis and success criterion for the next training attempt. The data tool selects and validates training examples while checking for leakage and formatting errors. The training tool uses LlamaFactory, an existing training library, to run a stable process that exports a ready to evaluate model. The evaluation tool runs real benchmarks and captures raw output and failure details, and the logging tool keeps a persistent record of experiments across long running sessions. Together these are meant to reduce common problems in unsupervised training runs, like broken data formats, wrong chat templates, and lost experiment history. This branch of the repository contains only the instruction files and reusable skills, meant to be copied into a Codex or OpenCode agent setup. The complete benchmark runner, agent wrappers, and full pipeline live on a separate branch called full-code, which users are told to check out if they want to run the entire evaluation process rather than just reuse the planning and training instructions.