Searcharvester banner
vakovalskii vakovalskii

Searcharvester

DevOps community

Description

Self-hosted search + markdown harvester for AI agents. SearXNG (100+ engines) + FastAPI + trafilatura. Tavily-compatible /search plus /extract with size presets and pagination. One-command Docker Compose.

Installation

This entry records only its repository, not the path inside it, so there is no exact command to give. Open the source below and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

README

Searcharvester 🌾

**Self-hosted search + extract + deep research for AI agents**

📖 **Docs:** [English](docs/en/README.md) · [Русский](docs/ru/README.md) · [中文](docs/zh/README.md)

Three composable HTTP services in a single `docker compose up`:

  • /search — Tavily-compatible search via SearXNG (100+ engines)
  • /extract — URL → clean markdown via trafilatura, with size presets and pagination
  • /research — deep research agent: give it a question, get back a cited markdown report

No API keys, no quotas, fully self-hosted. Pre-built image on GHCR.

🚀 Quick start

# 1. Clone
git clone git@github.com:vakovalskii/searcharvester.git
cd searcharvester

# 2. Config
cp config.example.yaml config.yaml
# Change server.secret_key (32+ chars)

# 3. (Optional) LLM credentials for /research — any OpenAI-compatible endpoint
cat > .env <

🧱 Three services, one API

1️⃣ `POST /search` — Tavily-compatible search

Drop-in replacement for the [Tavily](https://tavily.com) API:

from tavily import TavilyClient
client = TavilyClient(api_key="ignored", base_url="http://localhost:8000")
response = client.search(query="...", max_results=5, include_raw_content=True)