What is auto-lable?

Auto-Label is a self-hosted platform for labeling images to train computer vision models, aimed at teams building datasets for tasks like text recognition, segmentation, and object detection. It runs as a local web app with a Flask backend and a browser based interface, and it combines five modules: text recognition, segmentation, object detection, rotated object detection, and model training. For text recognition, it runs two OCR engines side by side. When both engines agree on a result, the image is automatically pre-labeled, when they disagree, it is flagged for a human to review and correct, with a dashboard tracking totals, pending reviews, and corrected results. For segmentation, it uses a model called SAM3, which can be guided with positive and negative points, boxes, or typed text prompts, plus manual polygon and circle drawing, draggable masks, multi-object support per image, and undo up to 50 steps. A tool called LocateAnything can detect objects by category name or free text phrase and hand the resulting boxes straight to SAM3 for segmentation. The object detection module reuses LocateAnything for category and phrase based detection alongside manual box and polygon tools. Rotated object detection lets you trace a polygon around an object, automatically fits the smallest enclosing rotated rectangle, and can also derive rotated boxes from SAM3 masks, with a YOLOv8-OBB model available for automatic batch predictions. The fifth module handles training: you can configure a training run, including data folder, epochs, image size, and batch size, directly in the browser, and it trains in the background without blocking labeling work, showing live progress and logs. Setup requires Windows, multiple separate Python environments for the main service, the SAM3 service, and the LocateAnything service, an NVIDIA GPU with at least 8GB of memory, and several gigabytes of downloaded model weights. The main server, SAM3 service, and LocateAnything service run as three separate processes that talk to each other over HTTP. Output is written as JSON or JSONL files per image, plus standard training artifacts like model weights and performance charts.