whatisgithub

What is egobabyvlm?

facebookresearch/egobabyvlm — explained in plain English

Analysis updated 2026-07-17 · repo last pushed 2026-06-23

★ 7PythonAudience · researcherComplexity · 5/5ActiveSetup · hard

In one sentence

A research challenge and toolkit for training AI vision-language models on 863 hours of head-mounted infant video, testing whether models can learn from messy real-world data instead of curated web datasets.

Mindmap

mindmap
  root((egobabyvlm))
    Inputs
      Egocentric baby video
      Accompanying speech
    Outputs
      Trained vision-language model
      Leaderboard scores
    Use Cases
      Benchmark vision-language learning
      Study embodied AI
      Test developmental learning
    Tech Stack
      Python
      PyTorch
      Hydra
      Pixi

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Train a visual-language model exclusively on egocentric infant video and speech to test naturalistic learning.

USE CASE 2

Evaluate a trained model on vision-only, language-only, and cross-modal benchmark tasks.

USE CASE 3

Use the four reference approaches (contrastive learning, language modeling, DINOv2-based vision, LLaVA-style generative model) as starting points for research.

USE CASE 4

Generate adversarial test cases to probe subtle failures in a model's language understanding.

What is it built with?

PythonPyTorchHydraPixi

How does it compare?

facebookresearch/egobabyvlmbettyguo/local-deep-researchcaptaingrock/krea2trainer
Stars777
LanguagePythonPythonPython
Last pushed2026-06-23——
MaintenanceActive——
Setup difficultyhard—hard
Complexity5/5—4/5
Audienceresearcherdeveloperdesigner

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · hard Time to first run · 1day+

Requires processing 863 hours of video data and a Pixi-managed reproducible environment with PyTorch.

License is not stated in the available content.

Copy-paste prompts

Prompt 1
Set up egobabyvlm's Pixi environment and run the contrastive learning baseline on the egocentric baby video dataset.
Prompt 2
Explain how egobabyvlm evaluates cross-modal understanding by matching baby-video images to text descriptions.
Prompt 3
Help me use egobabyvlm's benchmark generation tools to create test cases for comparatives and spatial relationships.
Prompt 4
Walk me through egobabyvlm's preprocessing pipeline that turns raw egocentric video into training data.

Frequently asked questions

What is egobabyvlm?

A research challenge and toolkit for training AI vision-language models on 863 hours of head-mounted infant video, testing whether models can learn from messy real-world data instead of curated web datasets.

What language is egobabyvlm written in?

Mainly Python. The stack also includes Python, PyTorch, Hydra.

Is egobabyvlm actively maintained?

Active — commit in last 30 days (last push 2026-06-23).

What license does egobabyvlm use?

License is not stated in the available content.

How hard is egobabyvlm to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is egobabyvlm for?

Mainly researcher.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.

What are similar repos?