How to Use SearchHive with Python

SearchHive provides a Python SDK that gives you programmatic web search, scraping, and AI-powered extraction through three APIs: SwiftSearch (web search), ScrapeForge (web scraping), and DeepDive (structured data extraction). This guide walks through setup, each API, and real-world usage patterns.

Key Takeaways

• Install with pip: pip install searchhive — one package, all three APIs
• SwiftSearch returns structured free JSON formatter search results with titles, URLs, and snippets
• ScrapeForge extracts clean content from any URL — handles JavaScript, CAPTCHAs, and proxies
• DeepDive uses AI to extract structured data from unstructured web content
• Free tier available — no credit card needed to start

---

How do I install and set up SearchHive?

Install the SDK from PyPI:

pip install searchhive

Get your API key from the SearchHive dashboard. You can create a free account without a credit card.

Initialize the client:

from searchhive import SwiftSearch, ScrapeForge, DeepDive

# Initialize with your API key
search = SwiftSearch(api_key='sh-your-api-key')
scraper = ScrapeForge(api_key='sh-your-api-key')
deeper = DeepDive(api_key='sh-your-api-key')

# Or use environment variable
import os
# export SEARCHHIVE_API_KEY=sh-your-api-key
search = SwiftSearch(api_key=os.environ['SEARCHHIVE_API_KEY'])

How do I search the web with SwiftSearch?

SwiftSearch is the simplest way to get web search results in Python. Pass a query, get structured JSON back:

from searchhive import SwiftSearch

search = SwiftSearch(api_key='your-api-key')

# Basic search
results = search.query('machine learning frameworks 2026')

for r in results:
    print(f"Title: {r['title']}")
    print(f"URL: {r['url']}")
    print(f"Snippet: {r['snippet']}")
    print('---')

With parameters for filtering:

# Search with filters
results = search.query(
    query='Python async frameworks',
    count=20,           # Number of results
    country='us',       # Geographic market
    language='en',      # Language
    date_range='month'  # Recent results only
)

Each result includes:

• title — the page title
• url — the full URL
• snippet — a text excerpt from the page
• domain — the source domain

How do I scrape web pages with ScrapeForge?

ScrapeForge extracts content from any URL. It handles JavaScript rendering, CAPTCHAs, and proxy rotation automatically:

from searchhive import ScrapeForge

scraper = ScrapeForge(api_key='your-api-key')

# Scrape a single page
result = scraper.scrape('https://example.com/blog-post')

print(result['title'])       # Page title
print(result['markdown'])    # Content as clean markdown
print(result['text'])        # Plain text (stripped markdown)
print(result['links'])       # All links on the page

Scrape multiple pages in a batch:

urls = [
    'https://example.com/page-1',
    'https://example.com/page-2',
    'https://example.com/page-3',
]

results = scraper.scrape_batch(urls)  # Concurrent scraping
for result in results:
    print(f"{result['title']}: {len(result['markdown'])} chars")

ScrapeForge handles:

• JavaScript rendering — pages that require JS to load content
• CAPTCHA solving — automated CAPTCHA detection and resolution
• Proxy rotation — built-in IP rotation to avoid blocks
• Rate limiting — respects rate limits and retries automatically

How do I extract structured data with DeepDive?

DeepDive uses AI to pull structured data from web content. You tell it what fields you want, and it extracts them:

from searchhive import DeepDive

deeper = DeepDive(api_key='your-api-key')

# Extract structured data from raw content
content = """
Apple Inc. reported Q4 2025 revenue of $94.9 billion, up 6% year-over-year.
CEO Tim Cook attributed growth to strong iPhone 16 sales and services revenue.
The company's stock closed at $198.50 on NASDAQ.
"""

result = deeper.extract(
    content=content,
    schema={
        'company': 'str',
        'quarter': 'str',
        'revenue': 'str',
        'growth_percent': 'float',
        'ceo': 'str',
        'stock_price': 'float'
    }
)

print(result)
# {'company': 'Apple Inc.', 'quarter': 'Q4 2025', 'revenue': '$94.9 billion',
#  'growth_percent': 6.0, 'ceo': 'Tim Cook', 'stock_price': 198.50}

Extract from a URL directly:

result = deeper.extract_url(
    url='https://news.example.com/apple-earnings',
    schema={'headline': 'str', 'date': 'str', 'summary': 'str', 'entities': 'list[str]'}
)

How do I combine all three APIs?

The real power of SearchHive is combining search, scraping, and extraction into one pipeline:

from searchhive import SwiftSearch, ScrapeForge, DeepDive

search = SwiftSearch(api_key='your-key')
scraper = ScrapeForge(api_key='your-key')
deeper = DeepDive(api_key='your-key')

# Step 1: Find relevant pages
results = search.query('best SaaS pricing strategies 2026', count=10)

# Step 2: Scrape full content from top results
articles = []
for r in results[:5]:
    page = scraper.scrape(r['url'])
    articles.append({
        'title': page['title'],
        'content': page['markdown'],
        'source': r['url']
    })

# Step 3: Extract structured insights from each article
insights = []
for article in articles:
    data = deeper.extract(
        content=article['content'],
        schema={
            'main_strategy': 'str',
            'pricing_models_mentioned': 'list[str]',
            'key_takeaway': 'str'
        }
    )
    insights.append({
        'source': article['source'],
        'title': article['title'],
        **data
    })

# Output consolidated research
for i in insights:
    print(f"\n📌 {i['title']}")
    print(f"   Strategy: {i['main_strategy']}")
    print(f"   Models: {', '.join(i['pricing_models_mentioned'])}")
    print(f"   Takeaway: {i['key_takeaway']}")

This search → scrape → extract pattern is the foundation for automated research, content monitoring, competitive analysis, and AI agent tooling. For more on the AI agent use case, see How to Build an AI Agent with Web Access.

How do I handle errors?

The SDK provides clear error handling:

from searchhive import SwiftSearch, SearchHiveError, RateLimitError

search = SwiftSearch(api_key='your-key')

try:
    results = search.query('test query')
except RateLimitError as e:
    # Too many requests — wait and retry
    print(f"Rate limited. Retry after {e.retry_after}s")
except SearchHiveError as e:
    # Other API errors
    print(f"API error: {e.message}")
except Exception as e:
    # Network or unexpected errors
    print(f"Unexpected error: {e}")

The SDK also supports async for high-throughput applications:

import asyncio
from searchhive import AsyncSwiftSearch

async def search_multiple():
    search = AsyncSwiftSearch(api_key='your-key')
    queries = ['Python async', 'Rust web', 'Go concurrency']
    
    tasks = [search.query(q) for q in queries]
    results = await asyncio.gather(*tasks)
    
    for query, result in zip(queries, results):
        print(f"{query}: {len(result)} results")

asyncio.run(search_multiple())

How does SearchHive compare to alternatives?

SearchHive's advantage is the unified SDK — one package, one API key, three capabilities. Competitors typically specialize in one area (search OR scraping), requiring multiple subscriptions and integrations.

Summary

SearchHive's Python SDK gives you web search, scraping, and AI-powered extraction through a single interface. Install with pip install searchhive, initialize with your API key, and start building search → scrape → extract pipelines. The SDK supports both sync and async, handles errors gracefully, and includes a free tier for testing.

Get your API key at searchhive.dev/docs and start building.

---

Related reading: What Is SwiftSearch API | How to Handle Rate Limiting in Web Scraping | Compare SearchHive vs ScraperAPI