How to Use SearchHive with Python
SearchHive provides a Python SDK that gives you programmatic web search, scraping, and AI-powered extraction through three APIs: SwiftSearch (web search), ScrapeForge (web scraping), and DeepDive (structured data extraction). This guide walks through setup, each API, and real-world usage patterns.
Key Takeaways
- Install with pip:
pip install searchhive— one package, all three APIs - SwiftSearch returns structured free JSON formatter search results with titles, URLs, and snippets
- ScrapeForge extracts clean content from any URL — handles JavaScript, CAPTCHAs, and proxies
- DeepDive uses AI to extract structured data from unstructured web content
- Free tier available — no credit card needed to start
How do I install and set up SearchHive?
Install the SDK from PyPI:
pip install searchhive
Get your API key from the SearchHive dashboard. You can create a free account without a credit card.
Initialize the client:
from searchhive import SwiftSearch, ScrapeForge, DeepDive
# Initialize with your API key
search = SwiftSearch(api_key='sh-your-api-key')
scraper = ScrapeForge(api_key='sh-your-api-key')
deeper = DeepDive(api_key='sh-your-api-key')
# Or use environment variable
import os
# export SEARCHHIVE_API_KEY=sh-your-api-key
search = SwiftSearch(api_key=os.environ['SEARCHHIVE_API_KEY'])
How do I search the web with SwiftSearch?
SwiftSearch is the simplest way to get web search results in Python. Pass a query, get structured JSON back:
from searchhive import SwiftSearch
search = SwiftSearch(api_key='your-api-key')
# Basic search
results = search.query('machine learning frameworks 2026')
for r in results:
print(f"Title: {r['title']}")
print(f"URL: {r['url']}")
print(f"Snippet: {r['snippet']}")
print('---')
With parameters for filtering:
# Search with filters
results = search.query(
query='Python async frameworks',
count=20, # Number of results
country='us', # Geographic market
language='en', # Language
date_range='month' # Recent results only
)
Each result includes:
title— the page titleurl— the full URLsnippet— a text excerpt from the pagedomain— the source domain
How do I scrape web pages with ScrapeForge?
ScrapeForge extracts content from any URL. It handles JavaScript rendering, CAPTCHAs, and proxy rotation automatically:
from searchhive import ScrapeForge
scraper = ScrapeForge(api_key='your-api-key')
# Scrape a single page
result = scraper.scrape('https://example.com/blog-post')
print(result['title']) # Page title
print(result['markdown']) # Content as clean markdown
print(result['text']) # Plain text (stripped markdown)
print(result['links']) # All links on the page
Scrape multiple pages in a batch:
urls = [
'https://example.com/page-1',
'https://example.com/page-2',
'https://example.com/page-3',
]
results = scraper.scrape_batch(urls) # Concurrent scraping
for result in results:
print(f"{result['title']}: {len(result['markdown'])} chars")
ScrapeForge handles:
- JavaScript rendering — pages that require JS to load content
- CAPTCHA solving — automated CAPTCHA detection and resolution
- Proxy rotation — built-in IP rotation to avoid blocks
- Rate limiting — respects rate limits and retries automatically
How do I extract structured data with DeepDive?
DeepDive uses AI to pull structured data from web content. You tell it what fields you want, and it extracts them:
from searchhive import DeepDive
deeper = DeepDive(api_key='your-api-key')
# Extract structured data from raw content
content = """
Apple Inc. reported Q4 2025 revenue of $94.9 billion, up 6% year-over-year.
CEO Tim Cook attributed growth to strong iPhone 16 sales and services revenue.
The company's stock closed at $198.50 on NASDAQ.
"""
result = deeper.extract(
content=content,
schema={
'company': 'str',
'quarter': 'str',
'revenue': 'str',
'growth_percent': 'float',
'ceo': 'str',
'stock_price': 'float'
}
)
print(result)
# {'company': 'Apple Inc.', 'quarter': 'Q4 2025', 'revenue': '$94.9 billion',
# 'growth_percent': 6.0, 'ceo': 'Tim Cook', 'stock_price': 198.50}
Extract from a URL directly:
result = deeper.extract_url(
url='https://news.example.com/apple-earnings',
schema={'headline': 'str', 'date': 'str', 'summary': 'str', 'entities': 'list[str]'}
)
How do I combine all three APIs?
The real power of SearchHive is combining search, scraping, and extraction into one pipeline:
from searchhive import SwiftSearch, ScrapeForge, DeepDive
search = SwiftSearch(api_key='your-key')
scraper = ScrapeForge(api_key='your-key')
deeper = DeepDive(api_key='your-key')
# Step 1: Find relevant pages
results = search.query('best SaaS pricing strategies 2026', count=10)
# Step 2: Scrape full content from top results
articles = []
for r in results[:5]:
page = scraper.scrape(r['url'])
articles.append({
'title': page['title'],
'content': page['markdown'],
'source': r['url']
})
# Step 3: Extract structured insights from each article
insights = []
for article in articles:
data = deeper.extract(
content=article['content'],
schema={
'main_strategy': 'str',
'pricing_models_mentioned': 'list[str]',
'key_takeaway': 'str'
}
)
insights.append({
'source': article['source'],
'title': article['title'],
**data
})
# Output consolidated research
for i in insights:
print(f"\n📌 {i['title']}")
print(f" Strategy: {i['main_strategy']}")
print(f" Models: {', '.join(i['pricing_models_mentioned'])}")
print(f" Takeaway: {i['key_takeaway']}")
This search → scrape → extract pattern is the foundation for automated research, content monitoring, competitive analysis, and AI agent tooling. For more on the AI agent use case, see How to Build an AI Agent with Web Access.
How do I handle errors?
The SDK provides clear error handling:
from searchhive import SwiftSearch, SearchHiveError, RateLimitError
search = SwiftSearch(api_key='your-key')
try:
results = search.query('test query')
except RateLimitError as e:
# Too many requests — wait and retry
print(f"Rate limited. Retry after {e.retry_after}s")
except SearchHiveError as e:
# Other API errors
print(f"API error: {e.message}")
except Exception as e:
# Network or unexpected errors
print(f"Unexpected error: {e}")
The SDK also supports async for high-throughput applications:
import asyncio
from searchhive import AsyncSwiftSearch
async def search_multiple():
search = AsyncSwiftSearch(api_key='your-key')
queries = ['Python async', 'Rust web', 'Go concurrency']
tasks = [search.query(q) for q in queries]
results = await asyncio.gather(*tasks)
for query, result in zip(queries, results):
print(f"{query}: {len(result)} results")
asyncio.run(search_multiple())
How does SearchHive compare to alternatives?
| Feature | SearchHive | SerpAPI | ScraperAPI | Bright Data |
|---|---|---|---|---|
| Web Search | ✅ SwiftSearch | ✅ Yes | ❌ No | ✅ SERP API |
| Web Scraping | ✅ ScrapeForge | ❌ No | ✅ Yes | ✅ Web Unlocker |
| AI Extraction | ✅ DeepDive | ❌ No | ❌ No | ❌ No |
| Python SDK | ✅ Official | ✅ Official | ✅ Official | ✅ Official |
| Async Support | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
| Free Tier | ✅ Yes | ✅ Limited | ✅ 1000 requests | ✅ Trial |
| JS Rendering | ✅ Built-in | N/A | ✅ Yes | ✅ Yes |
SearchHive's advantage is the unified SDK — one package, one API key, three capabilities. Competitors typically specialize in one area (search OR scraping), requiring multiple subscriptions and integrations.
Summary
SearchHive's Python SDK gives you web search, scraping, and AI-powered extraction through a single interface. Install with pip install searchhive, initialize with your API key, and start building search → scrape → extract pipelines. The SDK supports both sync and async, handles errors gracefully, and includes a free tier for testing.
Get your API key at searchhive.dev/docs and start building.
Related reading: What Is SwiftSearch API | How to Handle Rate Limiting in Web Scraping | Compare SearchHive vs ScraperAPI