What is background-matting?

This repository contains the code for a research project called Background Matting, published at the IEEE CVPR 2020 computer vision conference by researchers at the University of Washington. The project tackles a classic photography and video problem: how do you cleanly separate a person from their background so you can place them in front of a different scene? Traditional solutions use a green screen, which requires controlled studio lighting and a dedicated colored backdrop. Background Matting takes a different approach. Instead of a green screen, you capture two photos: one with the subject in frame, and one of the same background without the subject. The system compares these two images and uses a neural network to figure out precisely which pixels belong to the person, including difficult areas like hair, which would normally be hard to cut out cleanly. The output is called an alpha matte, which is a map that describes how transparent each pixel is at the boundary between the subject and background. With this matte, you can composite the subject onto any background you choose, with realistic edges that look natural rather than jagged or blurry. The code supports both photos and videos. For photos, you provide a subject image, a matching background image, and a target background. A setup step runs a segmentation model to roughly identify the person, then aligns the background image to the subject image to account for any camera movement between shots. Running the main script then produces the composited output. The repository also notes that a newer version of the project, Background Matting V2, has been released separately with better quality and real-time performance at high resolutions. The original version covered here is the initial research release and is intended for non-commercial use under a Creative Commons license.