Give an AI agent search and fetch tools through a self hosted API.
Convert any web page into clean Markdown text for an agent to read.
Prototype an agent's web research loop before adopting a hosted service.
Study a minimal, single file example of a search plus fetch API.
| laureldev/laurel.dev | 2arons/llm-cli | abe238/claude-video-plus | |
|---|---|---|---|
| Stars | 11 | 11 | 11 |
| Language | Python | Python | Python |
| Setup difficulty | moderate | easy | easy |
| Complexity | 2/5 | 2/5 | 3/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires creating a Google Custom Search Engine and API key before the endpoints work.
laurel.dev Core is a small, self hostable API that gives AI agents web search and page reading abilities through just two endpoints. A search endpoint takes a query and returns a list of relevant links with titles and short snippets. A fetch endpoint takes a single URL, downloads that page, and returns its main content converted into clean Markdown text, with ads, navigation, and clutter stripped out. The idea is that an agent first searches for sources, reads the short snippets to judge which ones matter, then only fetches the full page content for the links actually worth reading. Under the hood, the search endpoint calls Google's Custom Search JSON API, which requires a Google API key and a Programmable Search Engine set up by the person running the service. The fetch endpoint downloads a page and uses the readability library to pull out the readable article text before converting it to Markdown. The README describes the whole implementation as basically one file, api/main.py, and encourages readers to look at it directly rather than trust a black box. Running it locally requires Docker and Docker Compose, plus a Google API key and search engine id placed into an environment file. Once running, the API listens on port 8000 and can be called directly with curl or wired into an agent framework as two tools, one for searching and one for fetching, alongside instructions telling the model to search first and fetch only what it needs. The README is upfront that this repository is a deliberately minimal core: it does not include caching, rate limiting, quota management, billing, or the kind of source ranking tuned for heavy real world traffic. Google's own free tier caps searches at 100 per day, which the README says is fine for experimenting but not enough for production use. For anyone who wants those production features without building them, the project's authors point to a paid hosted version at laurel.dev that offers the same two endpoints, plus caching, unlimited flat-price search, and a hosted MCP endpoint. The code in this repository is released under the MIT license.
A minimal, self hostable API giving AI agents two tools, search and fetch, to find web sources and read clean page content.
Mainly Python. The stack also includes Python, FastAPI, Docker.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.