What is shap-e?

Shap-E is a research project from OpenAI that generates three-dimensional objects from text descriptions or images. You can type a prompt like "a chair that looks like an avocado" or "a birthday cupcake" and the model produces a 3D shape. You can also provide a photograph or rendered image and have the model generate a 3D version of the object shown. The approach is based on a research paper and works by learning a compact mathematical representation of 3D shapes, called implicit functions, which can be decoded into viewable 3D models. This is different from producing a mesh or a point cloud directly. The outputs can be displayed as rotating animated previews. Installing the library requires Python and pip. The repository includes three Jupyter notebooks to help people get started: one for text-to-3D generation, one for image-to-3D generation, and one that demonstrates encoding an existing 3D model back into the learned representation. The image-to-3D path works best when the background is removed from the input image first. The encoding notebook additionally requires Blender, a free open source 3D software application, to generate the image renders it needs as input. This is a research release, meaning it is intended to share the methods and models described in the accompanying paper rather than to serve as a finished product for end users.