What is samurai?

SAMURAI is a Python research tool from the University of Washington that can follow and outline a specific object as it moves through a video, frame by frame. You point it at an object in the first frame by drawing a bounding box around it, and the system tracks that object through the rest of the clip, drawing a detailed outline around it at each step. It does not need to be trained on your particular video or object type: it works immediately on whatever you give it, which is why the authors call it a zero-shot method. The system is built on top of SAM 2, a model released by Meta that can segment and identify objects in images and videos. SAMURAI adds a motion-aware memory layer on top of SAM 2 that helps it stay locked onto an object even when the object moves quickly, gets partially blocked, or changes appearance as it moves. The motion estimation uses a classical technique called a Kalman filter, which predicts where a moving object is likely to be next based on its past positions and speeds. To use it, you need a computer with a compatible GPU and Python 3.10 or higher. After installing the required libraries and downloading the model weights, you can run a demo script on any video file or folder of image frames by specifying the video path and a text file that gives the initial bounding box coordinates. The repository also includes scripts for evaluating performance against standard academic tracking benchmarks such as LaSOT, GOT-10k, and OTB100. Streaming or live webcam input is not currently supported. The tool is designed for pre-recorded video files. The code was produced as part of a research paper and is primarily intended for academic use. It is built on top of SAM 2, which is developed by Meta, and the repository includes citation instructions for both the original SAM 2 paper and the SAMURAI paper.