Animate a still image into a video clip by providing the image and a text description of the desired motion.
Replace or move an object in an existing video by drawing a mask around the region to edit.
Extend the edges of a video frame outward to create a wider shot from existing footage.
Make a character in a video follow the movements of a person in a reference clip.
| ali-vilab/vace | fastapi-admin/fastapi-admin | mapillary/opensfm | |
|---|---|---|---|
| Stars | 3,765 | 3,765 | 3,766 |
| Language | Python | Python | Python |
| Setup difficulty | hard | moderate | hard |
| Complexity | 5/5 | 3/5 | 4/5 |
| Audience | researcher | developer | researcher |
Figures from each repo's GitHub metadata at analysis time.
Requires Python 3.10, CUDA 12.4, PyTorch 2.5.1, and a recent Nvidia GPU, no CPU fallback available.
VACE is an AI model for video creation and editing released by Alibaba's Tongyi Lab. It was accepted at ICCV 2025, one of the top computer vision research conferences. The model handles a range of video tasks from a single unified system rather than requiring separate tools for each operation. The three core modes are: generating a video from a reference image or video clip (R2V), editing an existing video (V2V), and editing a specific region of a video defined by a mask (MV2V). In practice this covers things like moving an object within a scene, replacing one object with another, animating a still image, extending the edges of a video frame outward, or making a character in a video follow the motion of a reference person. These tasks can be combined in a single pipeline. Several model versions are available. The smaller ones (1.3B parameters) run at resolutions around 480x832 pixels across roughly 80 frames. The largest model (14B parameters) targets 720x1280 resolution. All models are hosted on HuggingFace and ModelScope, and the two underlying video generation systems they build on are Wan2.1 (from Alibaba) and LTX-Video (from Lightricks). Licenses vary by model: most of the Wan-based models are Apache-2.0, while the LTX-based model carries a separate RAIL-M license. Setup requires Python 3.10, CUDA 12.4, and PyTorch 2.5.1 or later, so a reasonably recent Nvidia GPU is needed. After installing dependencies, you download the model weights and run generation from the command line by specifying a task name, a video file, and a text prompt. A Gradio-based graphical interface is also included for interactive use. Preprocessing tools for extracting depth maps, pose information, or flow data are available as an optional separate install. The codebase also includes a benchmark dataset for evaluating the model's output quality across the supported task types.
VACE is an Alibaba AI model that generates and edits videos, animate a still image, swap objects in a scene, extend frame edges, or transfer motion from a reference clip, all from one unified system.
Mainly Python. The stack also includes Python, PyTorch, CUDA.
Most models are Apache-2.0 (free for commercial use), the LTX-based model uses a RAIL-M license with additional restrictions on certain uses.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.