How to Ai Agent Mcp Integration — Step-by-Step

MCP (Model Context Protocol) is the standard for connecting AI agents to external tools and data sources. If you want your AI agent to search the web, scrape pages, or extract data — instead of relying on stale training data — MCP integration is the way to do it.

This tutorial shows you how to set up MCP integration for AI agents using SearchHive as the web data provider. You will have an AI agent that can search, scrape, and research the web in real-time.

Key Takeaways

• MCP (Model Context Protocol) lets AI agents access external tools through a standardized interface
• SearchHive provides a native MCP server that gives AI agents access to SwiftSearch, ScrapeForge, and DeepDive
• Setup takes under 5 minutes: install the npm package, add your API key, configure your MCP client
• Works with Claude Desktop, Cline, Cursor, and any MCP-compatible client
• Free tier includes 500 credits — enough to build and test your agent integration

Prerequisites

Before starting, you need:

• Node.js 18+ (for the MCP server)
• An MCP-compatible AI client (Claude Desktop, Cline VS Code extension, or Cursor)
• A SearchHive API key (get one free — 500 credits, no CC)
• npm (usually included with Node.js)

Step 1: Understand What MCP Does for AI Agents

MCP solves a specific problem: AI models are trained on static data, but users need real-time information. Without MCP, an AI agent cannot:

• Look up current prices
• Read a competitor's website
• Find recent news articles
• Extract data from a specific URL

MCP provides a standardized protocol for the AI agent to call external tools. The agent decides when to use a tool, sends a request through MCP, and gets structured data back. SearchHive's MCP server exposes three capabilities:

1. swift_search — real-time web search (SwiftSearch)
2. scrape — scrape any URL (ScrapeForge)
3. deep_dive — extract structured data from pages (DeepDive)

Step 2: Install the SearchHive MCP Server

The MCP server is an npm package that runs locally and connects to SearchHive's API:

npm install @searchhive/mcp-server

You do not need to start it manually — your MCP client (Claude Desktop, Cline, etc.) will launch it automatically based on your configuration.

Step 3: Get Your SearchHive API Key

If you do not already have one:

1. Go to searchhive.dev/signup
2. Create a free account (no credit card)
3. You get 500 free credits immediately
4. Copy your API key from the dashboard

Step 4: Configure Claude Desktop

Claude Desktop is the most common MCP client. Edit your Claude Desktop configuration file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

Add the SearchHive MCP server configuration:

{
  "mcpServers": {
    "searchhive": {
      "command": "npx",
      "args": ["@searchhive/mcp-server"],
      "env": {
        "SEARCHHIVE_API_KEY": "your_api_key_here"
      }
    }
  }
}

Replace your_api_key_here with your actual API key. Restart Claude Desktop, and the SearchHive tools will be available.

Step 5: Configure Cline (VS Code Extension)

Cline is a popular AI coding assistant that supports MCP:

1. Install the Cline extension in VS Code
2. Open Cline settings (gear icon)
3. Go to MCP Servers
4. Click "Add MCP Server"
5. Enter:
   • Name: searchhive
   • Command: npx
   • Args: @searchhive/mcp-server
   • Env: SEARCHHIVE_API_KEY=your_api_key_here
6. Save and restart Cline

Step 6: Configure Cursor IDE

Cursor also supports MCP integration:

1. Open Cursor settings
2. Go to the MCP section
3. Add a new MCP server with the same configuration as Claude Desktop
4. The SearchHive tools will appear in Cursor's tool menu

Step 7: Use the MCP Tools in Your AI Agent

Once configured, your AI agent can use SearchHive tools automatically. Here are example prompts that trigger MCP usage:

Web Search: "Search for the latest Python web scraping libraries and compare their features"

The agent will call swift_search and return real-time results.

Page Scraping: "Go to https://competitor.com/pricing and extract all their plan details"

The agent will call scrape to get the page content, then extract the pricing information.

Deep Research: "Find the top 5 AI agent frameworks and summarize their documentation"

The agent will chain multiple MCP calls: search for frameworks, scrape their documentation pages, and synthesize the results.

Step 8: Programmatic MCP Usage with Python

If you are building your own AI agent with Python, you can also use SearchHive directly (bypassing MCP for lower latency):

from searchhive import SwiftSearch, ScrapeForge, DeepDive

class WebAgent:
    def __init__(self, api_key):
        self.search = SwiftSearch(api_key=api_key)
        self.scrape = ScrapeForge(api_key=api_key)
        self.extract = DeepDive(api_key=api_key)

    def research_topic(self, topic, num_sources=5):
        # Step 1: Search for relevant sources
        results = self.search.search(query=topic, num=num_sources)
        
        # Step 2: Scrape top results for full content
        articles = []
        for result in results.organic[:3]:
            page = self.scrape.scrape(result.url)
            articles.append({
                "title": result.title,
                "url": result.url,
                "content": page.content[:2000]  # First 2000 chars
            })
        
        return articles

    def extract_structured_data(self, url):
        # Extract structured data from any page
        data = self.extract.extract(url)
        return {
            "title": data.title,
            "author": data.author,
            "content": data.content,
            "headings": [h.text for h in data.headings]
        }

agent = WebAgent(api_key="your_api_key")

# Research a topic
articles = agent.research_topic("best practices for RAG systems 2025")
for article in articles:
    print(f"\n{article['title']}")
    print(f"  {article['url']}")
    print(f"  {len(article['content'])} chars extracted")

# Extract structured data from a specific page
data = agent.extract_structured_data("https://docs.example.com/api-reference")
print(f"\nExtracted {len(data['headings'])} headings")

Common Issues

MCP Server Not Starting

If Claude Desktop cannot start the MCP server, check:

1. Node.js is installed and accessible in your PATH
2. The npm package installed correctly: npm list @searchhive/mcp-server
3. Your API key is valid: test it with curl -H "Authorization: Bearer your_key" https://api.searchhive.dev/v1/search

Agent Not Using the Tools

Some AI clients need explicit permission to use MCP tools. Make sure:

1. The MCP server shows as "connected" in your client's status
2. You are using a model that supports tool use (Claude 3.5+ Sonnet, GPT-4, etc.)
3. Your prompt implies the need for web data — the agent decides when to call tools

Rate Limit Errors

If you hit rate limits, upgrade your SearchHive plan. The free tier has lower limits, and the Builder plan ($49/mo) provides 100,000 credits with higher throughput.

Next Steps

Now that your AI agent can access web data through MCP, here are ways to extend it:

• Build a competitor monitoring system with automated alerts
• Set up SPA scraping for JavaScript-heavy sites
• Learn automation for monitoring best practices

Get started free — grab your SearchHive API key with 500 credits and connect your first AI agent to the web in minutes. No credit card required.

Full MCP documentation is available at searchhive.dev/docs.