What is tooncrafter?

ToonCrafter is a research tool that fills in the missing motion between two cartoon images. You give it a starting frame and an ending frame, and it generates the frames in between, producing a short animated clip. It comes from researchers at CUHK and Tencent AI Lab, and was accepted at SIGGRAPH Asia 2024, one of the top academic venues for computer graphics. The core idea relies on a video diffusion model that was originally trained on real-world video. ToonCrafter adapts that model to understand cartoon-style images, so it can invent plausible motion even when the two input frames look very different. The model generates videos at 512x320 resolution, up to 16 frames per clip, and takes roughly 24 seconds to run on a high-end research GPU using 50 generation steps. Community members have brought the memory requirement down to around 10 GB through compression tricks, making it accessible on more consumer cards. Beyond basic interpolation, ToonCrafter supports two additional uses. First, you can supply sparse sketch frames as guidance, giving the model rough hints about what path the motion should follow between the two keyframes. Second, the model can colorize a sketch using a separate reference image, mapping the colors and style of the reference onto the outline drawing. Getting it running requires Python and a compatible GPU. The recommended setup uses Anaconda to create an isolated environment, install dependencies, and then either run a command-line script or launch a local Gradio web interface for a point-and-click experience. Pre-trained model weights are available on Hugging Face. The community has also built wrappers for ComfyUI, Google Colab notebooks, and a Windows-specific package, so there are multiple ways to get started depending on your setup. ToonCrafter is a research prototype, not a commercial product. The authors note that no official paid service or web application exists, so any third-party site charging for access is unofficial.