CrewAI Web Search Integration — Give Your Agents Internet Access

CrewAI has rapidly become one of the most popular multi-agent frameworks, with nearly 49,000 GitHub stars and a growing ecosystem of built-in tools. But your agents are only as good as the data they can access. Without web search integration, CrewAI agents operate in an information vacuum — unable to look up current prices, verify facts, or find the latest documentation.

This guide compares every web search option available for CrewAI agents, from built-in tools to third-party APIs, so you can pick the right approach for your use case.

Key Takeaways

• CrewAI ships with four built-in search tools: Serper, Brave, Exa, and Linkup
• Each built-in tool requires its own API key and has different pricing models
• SearchHive provides a single API key that covers search, scraping, and deep research
• For production CrewAI deployments, SearchHive's Builder plan ($49/mo for 100K credits) is significantly cheaper than running multiple search providers

CrewAI Web Search Tools Compared

Setting Up Built-in CrewAI Search Tools

Serper (Google Search)

from crewai import Agent, Task, Crew
from crewai_tools import SerperDevTool

# Set your API key
import os
os.environ["SERPER_API_KEY"] = "your-serper-key"

# Initialize the search tool
search_tool = SerperDevTool()

# Create a researcher agent with web search
researcher = Agent(
    role="Market Researcher",
    goal="Find current market data and competitor pricing",
    backstory="You are an expert at finding accurate, up-to-date business information.",
    tools=[search_tool],
    verbose=True
)

task = Task(
    description="Research the current pricing of AI search APIs",
    expected_output="A summary of pricing tiers from top 5 search API providers",
    agent=researcher
)

crew = Crew(agents=[researcher], tasks=[task])
result = crew.kickoff()
print(result)

Brave Search

from crewai_tools import BraveSearchTool

os.environ["BRAVE_API_KEY"] = "your-brave-key"

search_tool = BraveSearchTool()

analyst = Agent(
    role="News Analyst",
    goal="Find breaking news and current events",
    backstory="You monitor news sources and identify relevant stories.",
    tools=[search_tool],
    verbose=True
)

Exa (Neural Search)

from crewai_tools import ExaSearchTool

os.environ["EXA_API_KEY"] = "your-exa-key"

# Exa excels at semantic/conceptual search
search_tool = ExaSearchTool()

researcher = Agent(
    role="Technical Researcher",
    goal="Find in-depth technical documentation and papers",
    backstory="You search for detailed technical resources across the web.",
    tools=[search_tool],
    verbose=True
)

Using SearchHive with CrewAI (Recommended)

SearchHive gives you a single API that covers search, content scraping, and deep research — all under one key. This means you can equip different agents with different capabilities without managing five separate API accounts.

import requests
from crewai import Agent, Task, Crew
from crewai.tools import BaseTool

class SwiftSearchTool(BaseTool):
    name: str = "swift_search"
    description: str = "Search the web for current information. Use for factual lookups, pricing data, and news."

    def _run(self, query: str) -> str:
        response = requests.get(
            "https://api.searchhive.dev/v1/swift-search",
            headers={"Authorization": "Bearer YOUR_SEARCHHIVE_KEY"},
            params={"query": query, "limit": 5}
        )
        data = response.json()
        results = []
        for r in data.get("results", [])[:5]:
            results.append(f"- {r['title']}: {r['snippet']}")
        return "\n".join(results)

class DeepDiveTool(BaseTool):
    name: str = "deep_dive"
    description: str = "Get comprehensive analysis and extracted content from web pages. Use for in-depth research."

    def _run(self, query: str) -> str:
        response = requests.get(
            "https://api.searchhive.dev/v1/deep-dive",
            headers={"Authorization": "Bearer YOUR_SEARCHHIVE_KEY"},
            params={"query": query}
        )
        data = response.json()
        return data.get("summary", "No results found")

class ScrapeForgeTool(BaseTool):
    name: str = "scrape_forge"
    description: str = "Extract structured content from any URL. Provide the full URL to scrape."

    def _run(self, url: str) -> str:
        response = requests.post(
            "https://api.searchhive.dev/v1/scrape-forge",
            headers={"Authorization": "Bearer YOUR_SEARCHHIVE_KEY", "Content-Type": "application/json"},
            json={"url": url, "format": "markdown"}
        )
        data = response.json()
        return data.get("content", "Failed to scrape")[:3000]

# Build a research crew with all three capabilities
researcher = Agent(
    role="Senior Researcher",
    goal="Conduct thorough web research on any topic",
    backstory="You are a meticulous researcher who verifies facts and provides comprehensive analysis.",
    tools=[SwiftSearchTool(), DeepDiveTool(), ScrapeForgeTool()],
    verbose=True
)

writer = Agent(
    role="Technical Writer",
    goal="Turn research findings into clear, actionable reports",
    backstory="You write concise, accurate technical content.",
    tools=[ScrapeForgeTool()],
    verbose=True
)

research_task = Task(
    description="Research the latest developments in multi-agent AI frameworks",
    expected_output="A detailed research brief covering recent updates, pricing changes, and new features",
    agent=researcher
)

writing_task = Task(
    description="Write a 500-word executive summary from the research brief",
    expected_output="A polished executive summary document",
    agent=writer,
    context=[research_task]
)

crew = Crew(agents=[researcher, writer], tasks=[research_task, writing_task])
result = crew.kickoff()

CrewAI MCP Integration

CrewAI v1.14+ supports MCP (Model Context Protocol) servers natively. This means any search tool that exposes an MCP server — including SearchHive — can be plugged in without writing custom tool classes:

from crewai import Agent, Task, Crew

# Configure MCP server connection
mcp_config = {
    "mcpServers": {
        "searchhive": {
            "command": "npx",
            "args": ["-y", "@searchhive/mcp-server"],
            "env": {"SEARCHHIVE_API_KEY": "your-key"}
        }
    }
}

agent = Agent(
    role="Research Agent",
    goal="Search the web and analyze results",
    tools=[],  # MCP tools auto-discovered
    llm="claude-3-5-sonnet",
    verbose=True
)

Pricing Comparison for CrewAI Teams

If you're running a multi-agent crew with 3-5 agents that all need web access, costs multiply fast:

SearchHive covers search, scraping, and deep research under one credit system. At $9/month for 5,000 credits, you get capabilities that would otherwise require 2-3 separate API subscriptions.

Verdict

For CrewAI web search integration, the right choice depends on your needs:

• Quick prototyping: Use the built-in Serper tool — fastest setup, decent free tier
• Semantic search: Exa's built-in tool is excellent for conceptual queries
• Production multi-agent systems: SearchHive — single API key, search + scraping + deep research, and 90% lower cost than running multiple search providers. The Builder plan at $49/mo gives you 100K credits, which handles serious multi-agent workloads

The main advantage of SearchHive for CrewAI is that different agents in the same crew can use different capabilities — one agent searches, another scrapes pages, a third runs deep analysis — all from the same API key and credit pool. No juggling separate Serper, Exa, and scraping subscriptions.

Get started with SearchHive's free tier — 500 credits with no credit card required. Check the API docs for integration examples.

---

Compare with other options: /compare/serpapi | /compare/tavily | /compare/brave-search-api