whatisgithub

What is deepresearch-datagen-cli?

oqura-ai/deepresearch-datagen-cli — explained in plain English

Analysis updated 2026-05-18

★ 40PythonAudience · dataComplexity · 3/5Setup · moderate

In one sentence

A Python command-line tool that automates building structured training datasets for LLMs by researching a topic across the web.

Mindmap

mindmap
  root((deepresearch datagen cli))
    What it does
      Builds training datasets
      Splits topics into subtopics
    Tech Stack
      Python
      LangChain
      LangGraph
      OpenAI
      Tavily
    Use Cases
      Dataset generation for finetuning
      Automated web research
    Audience
      Data teams
      Developers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Automatically build a structured training dataset on a specific topic for finetuning an LLM.

USE CASE 2

Break a broad research topic into subtopics and gather web sourced information for each.

USE CASE 3

Control research depth and results per section to match your dataset size needs.

What is it built with?

PythonLangChainLangGraphOpenAITavily

How does it compare?

oqura-ai/deepresearch-datagen-clicortex-ai-quant/crypto-arbitrage-bot-automated-tradingdexmal/realtime-vla-flash
Stars404040
LanguagePythonPythonPython
Setup difficultymoderatehardhard
Complexity3/53/55/5
Audiencedatageneralresearcher

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 30min

Requires Python 3.9+, an OpenAI API key, and a Tavily API key.

So what is it?

deepresearch-datagen-cli is a Python command-line tool that automates the process of building structured datasets for training AI language models. The problem it solves is the tedious manual work of gathering, organizing, and formatting data. Instead, you describe the kind of dataset you want, and the tool takes it from there. It breaks your request into focused subtopics, assigns a research agent to each one, has each agent search the web and extract relevant information, then merges everything into a single clean dataset file saved to your machine. You configure the depth of research, how many results to gather per section, and the AI model to use. It is built on LangChain and LangGraph, uses OpenAI as its AI backbone, and Tavily for web search. You would use this when you need a training dataset on a specific topic but do not want to spend hours manually collecting and structuring the data yourself. It requires Python 3.9 or newer, an OpenAI API key, and a Tavily API key to run.

Copy-paste prompts

Prompt 1
Use deepresearch-datagen-cli to generate a dataset about renewable energy policy for finetuning.
Prompt 2
Help me configure the research depth and per-section result count in deepresearch-datagen-cli.
Prompt 3
Explain how deepresearch-datagen-cli merges multiple subtopic research agents into one dataset file.

Frequently asked questions

What is deepresearch-datagen-cli?

A Python command-line tool that automates building structured training datasets for LLMs by researching a topic across the web.

What language is deepresearch-datagen-cli written in?

Mainly Python. The stack also includes Python, LangChain, LangGraph.

How hard is deepresearch-datagen-cli to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is deepresearch-datagen-cli for?

Mainly data.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.

What are similar repos?