OpenClaw Web Search, Scraping & Research Integration

Add real-time web search, web scraping, and multi-source research capabilities to your OpenClaw AI agent. A direct Brave Search API replacement with built-in web crawling, content extraction, and LLM-optimized output — all in three bash scripts with zero dependencies.

SearchHive provides AI agents with live internet access. Unlike Brave Search or SerpAPI, SearchHive combines web search with full-page content extraction and scraping in a single API. Built specifically for autonomous agents, LLMs, and AI workflows.

Brave Search API Alternative

Drop-in replacement for Brave Search API, SerpAPI, Google Custom Search, and Tavily. Real-time search results with clean, structured JSON output ready for LLM consumption.

Web Scraping & Crawling Built-In

Unlike search-only APIs, SearchHive includes full-page web scraping with JavaScript rendering, content extraction, and automatic text cleaning — no separate scraping service needed.

LLM-Optimized Output

All content is cleaned, deduplicated, and formatted as markdown with word counts and reading times. Token-efficient output reduces LLM context waste by up to 60%.

SearchHive vs Brave Search API vs SerpAPI

Feature	SearchHive	Brave Search	SerpAPI
Web Search
Web Scraping		—	—
Deep Research		—	—
LLM-Optimized Output		—	—
Free Tier			—
Zero Dependencies		—	—
Starting Price	Free	$3/1K queries	$50/month

Zero Dependencies — Pure Bash

No Python SDK, no Node.js package, no pip install. Just curl and jq. Works on any system with bash — Raspberry Pi, VPS, macOS, Linux. Perfect for resource-constrained AI agent deployments.

Three Scripts, Full Coverage

• SwiftSearch — Real-time web search (Google, Bing, DuckDuckGo)
• ScrapeForge — Full-page web scraping with content extraction
• DeepDive — Multi-source research with AI summarization

Token-Efficient for LLMs

Content is cleaned, deduplicated, and formatted as markdown. Boilerplate, navigation, footers, and ads are stripped automatically. Word counts and reading times included — reduce LLM context waste by up to 60%.

Built for AI Agents

Designed for autonomous AI agents, not humans. JSON output on stdout, errors on stderr, exit codes for success/failure. Credits-based billing means predictable costs for agentic workloads.

Quick Setup

1. Get your free API key

Sign up at searchhive.dev — free plan includes 5,000 credits/month (enough for ~5,000 searches or ~300 research queries).

2. Install the skill

# Copy into your OpenClaw workspace
cp -r integrations/openclaw/skills/searchhive \
  ~/.openclaw/workspace/skills/

# Set your API key
export SEARCHHIVE_API_KEY="sk_live_your_key_here"

# Make scripts executable
chmod +x ~/.openclaw/workspace/skills/searchhive/scripts/*.sh

# Test it
~/.openclaw/workspace/skills/searchhive/scripts/web_search.sh \
  "latest AI news" --count 3

3. Use in your agent

# Web search
./scripts/web_search.sh "best restaurants in Tokyo" --count 5

# URL scraping
./scripts/web_scrape.sh "https://example.com"

# Deep research
./scripts/web_research.sh "quantum computing breakthroughs" --pages 10

All scripts output JSON to stdout, errors to stderr. Exit code 0 on success.

API Response Format

SwiftSearch — Web Search Response

{
  "query": "AI agents 2025",
  "search_results": [
    {
      "title": "The Rise of Autonomous AI Agents",
      "link": "https://example.com/article",
      "snippet": "Cleaned, relevant snippet...",
      "position": 1
    }
  ],
  "credits_used": 1,
  "remaining_credits": 4999,
  "results_count": 10
}

ScrapeForge — Web Scraping Response

{
  "url": "https://example.com/article",
  "title": "Article Title",
  "text": "Cleaned markdown content...",
  "links": [...],
  "metadata": {"status_code": 200},
  "word_count": 1234,
  "reading_time": "6 min"
}

Requirements & Notes

Always use searchhive.dev — not www.searchhive.dev. The www subdomain can redirect and strip Authorization headers.
Requires jq for JSON construction. Install: apt install jq or brew install jq.
Free plan: 5,000 credits/month. SwiftSearch = 1 credit. ScrapeForge = 3 credits. DeepDive = ~15 credits.
Pricing: Free ($0) → Starter ($9/mo) → Builder ($49/mo) → Unicorn ($199/mo). No per-query overage fees.

Frequently Asked Questions

How is SearchHive different from Brave Search API?

Brave Search API only provides search results. SearchHive combines search with web scraping, content extraction, and deep research in a single API. All content is cleaned and optimized for LLM token efficiency. Plus, SearchHive has a free tier with 5,000 credits/month.

Can SearchHive replace Tavily or SerpAPI for my AI agent?

Yes. SearchHive is a direct replacement for Tavily, SerpAPI, Brave Search, and Google Custom Search. It provides real-time search results in the same JSON format, plus adds web scraping and content extraction that those APIs don't offer.

Does SearchHive work on Raspberry Pi or low-resource devices?

Yes. The OpenClaw integration uses pure bash scripts with curl and jq — no Python or Node.js runtime needed. It works on ARM devices, Raspberry Pi, and any system with bash.

What does "LLM-optimized" output mean?

SearchHive strips boilerplate (navigation, footers, ads, cookie banners), deduplicates content, normalizes whitespace, and formats output as clean markdown. This reduces token waste by up to 60% compared to raw search results or unprocessed scraped HTML.