What is agentcanvas?

AgentCanvas is a visual design platform for building embodied AI agents. These are AI systems that interact with physical or simulated environments, like robots navigating a room or answering questions about what they see. Instead of writing code, researchers draw node-and-wire graphs on a canvas. Each graph represents a complete agent: its perception, planning, memory, and actions all connected as data flowing from one node to the next. The core idea is simple: one JSON file equals one agent equals one graph. The platform targets several research areas. Vision-and-Language Navigation (VLN) involves agents following instructions to move through spaces. Embodied Question Answering (EQA) has agents explore environments to answer questions. Vision-Language-Action (VLA) covers policy benchmarks where agents decide what actions to take based on visual input. AgentCanvas works with multiple simulation environments, including Habitat-Sim, MatterSim, SAPIEN, and MuJoCo, so researchers can test agents in different virtual settings without rebuilding their entire setup each time. The system organizes functionality into swappable palettes: environment, method, model, and policy. Over 40 nodesets are available across these categories. Recent additions include wrappers for SLAM systems like VGGT-SLAM 2 and pySLAM, which handle mapping and camera tracking. The platform also supports 29 foundation models, including recent vision-language models and perception tools. A Graph SDK lets users build and run agents directly in Python, and the canvas editor supports editing node source code with hot-reload functionality. AgentCanvas serves two roles simultaneously. It acts as a harness that runs embodied agents against simulators in real time, and as a scaffold that coding agents can edit and verify. The same agent graph can run on a simulator today and, in principle, on a real robot later through environment nodesets and server mode. The project is a research preview from the Australian Institute for Machine Learning at the University of Adelaide, released under Apache 2.0. The public API is not yet frozen. The full README is longer than what was shown.