How to Scrape TikTok Data for Competitor Analysis

TikTok's algorithm and content trends move fast. For marketers, creators, and brands, tracking competitor performance on TikTok means understanding what content works, how engagement trends shift, and where opportunities exist. But TikTok's anti-scraping measures are aggressive -- rate limiting, login walls, and frequent DOM changes make direct scraping unreliable.

This guide covers realistic approaches to scraping TikTok data for competitor analysis using web APIs and the SearchHive platform.

Key Takeaways

• TikTok aggressively blocks direct scraping. Their anti-bot systems detect and block scrapers within minutes.
• Dedicated TikTok APIs (Apify, RapidAPI endpoints) provide structured data but cost $50-200/month for useful volumes.
• SearchHive ScrapeForge + DeepDive can extract public TikTok profile and video data at a lower cost for light-to-medium monitoring needs.
• Always respect TikTok's Terms of Service and use scraped data responsibly.

Prerequisites

• Python 3.8+
• requests library (pip install requests)
• A SearchHive API key (free signup with 500 credits)

Step 1: Understand TikTok's Anti-Scraping Defenses

Before writing any code, understand what you're dealing with:

1. Login walls -- Most TikTok pages require authentication or redirect to login if visited programmatically.
2. Rate limiting -- TikTok aggressively throttles requests per IP address.
3. Dynamic content loading -- Video data, comments, and engagement metrics load via API calls after the initial page render.
4. Signature validation -- TikTok's web app uses signed API parameters that are difficult to replicate.

Direct HTTP scraping of TikTok pages fails quickly:

# This will NOT work reliably
import requests
resp = requests.get("https://www.tiktok.com/@competitorhandle")
# Likely returns a login redirect or empty shell HTML

The viable approaches are:

• Use SearchHive ScrapeForge for publicly accessible profile/video pages
• Use dedicated TikTok data APIs for structured metrics
• Use TikTok's official Content Posting API (limited, requires partnership)

Step 2: Scrape Public TikTok Profiles with SearchHive

For publicly accessible TikTok profiles, SearchHive's ScrapeForge with JavaScript rendering can extract the visible page content:

import requests

API_KEY = "your-api-key"
BASE_URL = "https://api.searchhive.dev/v1"

def scrape_tiktok_profile(username):
    # Scrape a public TikTok profile page.
    # Args: username (without @)
    # Returns: dictionary with profile data
    response = requests.post(
        BASE_URL + "/scrape",
        headers={"Authorization": "Bearer " + API_KEY},
        json={
            "url": "https://www.tiktok.com/@" + username,
            "render_js": True,
            "format": "markdown"
        }
    )
    
    if response.status_code == 200:
        return response.json()
    else:
        raise Exception("Failed: " + str(response.status_code) + " - " + response.text)

profile = scrape_tiktok_profile("competitorhandle")
print(profile.get("markdown", profile.get("content", ""))[:1000])

Extract Structured Profile Data with DeepDive

For clean structured data instead of raw markdown:

def extract_profile_data(username):
    # Extract structured profile data using AI.
    response = requests.post(
        BASE_URL + "/deepdive",
        headers={"Authorization": "Bearer " + API_KEY},
        json={
            "url": "https://www.tiktok.com/@" + username,
            "prompt": (
                "Extract this TikTok profile data: display name, username, "
                "bio text, follower count, following count, like count, "
                "and the titles of the most recent 10 visible videos. "
                "Return as a structured JSON object."
            )
        }
    )
    return response.json()

data = extract_profile_data("competitorhandle")
print("Name: " + str(data.get("display_name", "N/A")))
print("Followers: " + str(data.get("follower_count", "N/A")))
print("Recent videos:")
for v in data.get("recent_videos", [])[:5]:
    print("  - " + str(v))

Step 3: Track TikTok Hashtag Trends

Monitoring hashtags tells you what's trending in your niche. Use SearchHive's SwiftSearch to find trending content:

def search_tiktok_hashtag(hashtag):
    # Search for content related to a TikTok hashtag.
    # Uses web search to find recent trending videos.
    response = requests.post(
        BASE_URL + "/search",
        headers={"Authorization": "Bearer " + API_KEY},
        json={
            "query": "tiktok #" + hashtag + " trending 2025",
            "num_results": 10
        }
    )
    return response.json()

results = search_tiktok_hashtag("webdevelopment")
for r in results.get("results", [])[:5]:
    print(r.get("title", "N/A"))
    print("  " + r.get("url", "N/A"))
    print()

Step 4: Analyze Competitor Content Strategy

Build a complete competitor analysis by combining multiple data sources:

import json
from datetime import datetime

class TikTokCompetitorAnalyzer:
    def __init__(self, api_key):
        self.api_key = api_key
        self.base_url = "https://api.searchhive.dev/v1"
        self.headers = {"Authorization": "Bearer " + api_key}
    
    def get_profile(self, username):
        # Get structured profile data.
        response = requests.post(
            self.base_url + "/deepdive",
            headers=self.headers,
            json={
                "url": "https://www.tiktok.com/@" + username,
                "prompt": (
                    "Extract: display name, bio, follower count, following count, "
                    "total likes, and list the most recent 10 video titles with "
                    "their view counts and like counts if visible."
                )
            }
        )
        return response.json()
    
    def get_content_themes(self, username):
        # Analyze the themes and topics in a creator's content.
        response = requests.post(
            self.base_url + "/deepdive",
            headers=self.headers,
            json={
                "url": "https://www.tiktok.com/@" + username,
                "prompt": (
                    "Based on the visible videos and bio, identify the top 5 "
                    "content themes or categories this creator posts about. "
                    "For each theme, estimate what percentage of their content "
                    "it represents. Return as a JSON array."
                )
            }
        )
        return response.json()
    
    def find_influencer_collaborations(self, username):
        # Search for known collaborations or mentions.
        response = requests.post(
            self.base_url + "/search",
            headers=self.headers,
            json={
                "query": username + " tiktok collab collaboration",
                "num_results": 5
            }
        )
        return response.json()
    
    def full_analysis(self, username):
        # Run complete competitor analysis.
        print("Analyzing @" + username + "...")
        
        profile = self.get_profile(username)
        themes = self.get_content_themes(username)
        collabs = self.find_influencer_collaborations(username)
        
        report = {
            "username": username,
            "analyzed_at": datetime.now().isoformat(),
            "profile": profile,
            "content_themes": themes,
            "collaborations": collabs
        }
        
        return report


analyzer = TikTokCompetitorAnalyzer(API_KEY)
report = analyzer.full_analysis("competitorhandle")
print(json.dumps(report, indent=2))

Step 5: Monitor Competitors Over Time

Set up regular monitoring to track growth and content changes:

import json
from pathlib import Path
from datetime import datetime

SNAPSHOTS_DIR = Path("./tiktok_snapshots")

def track_competitor(username, api_key):
    # Capture a competitor snapshot and compare with previous data.
    SNAPSHOTS_DIR.mkdir(exist_ok=True)
    
    analyzer = TikTokCompetitorAnalyzer(api_key)
    current = analyzer.get_profile(username)
    
    safe_name = username.replace("@", "").replace(" ", "_")
    snapshot_file = SNAPSHOTS_DIR / (safe_name + ".json")
    
    changes = []
    
    if snapshot_file.exists():
        with open(snapshot_file) as f:
            previous = json.load(f)
        
        # Compare follower count
        prev_followers = previous.get("profile", {}).get("follower_count")
        curr_followers = current.get("follower_count")
        
        if prev_followers and curr_followers:
            try:
                prev_num = int(str(prev_followers).replace(",", "").replace("K", "000").replace("M", "000000"))
                curr_num = int(str(curr_followers).replace(",", "").replace("K", "000").replace("M", "000000"))
                delta = curr_num - prev_num
                if abs(delta) > 0:
                    changes.append("Follower change: " + str(delta) + " (" + str(prev_followers) + " -> " + str(curr_followers) + ")")
            except (ValueError, AttributeError):
                pass
    
    # Save current snapshot
    with open(snapshot_file, "w") as f:
        json.dump({
            "username": username,
            "timestamp": datetime.now().isoformat(),
            "profile": current
        }, f, indent=2)
    
    if changes:
        print("Changes detected for @" + username + ":")
        for c in changes:
            print("  " + c)
    else:
        print("No significant changes for @" + username)
    
    return changes

Step 6: Alternative -- Dedicated TikTok APIs

For high-volume, reliable TikTok data extraction, dedicated APIs are more robust than web scraping:

These specialized tools handle TikTok's authentication and anti-scraping measures specifically. SearchHive is a better choice when you need general-purpose web scraping alongside TikTok data, or when you want to minimize the number of API subscriptions you manage.

Common Issues

TikTok serving login walls

If SearchHive returns login page content instead of profile data, the page has been restricted. Options:

• Try the profile URL with a trailing slash
• Use the TikTok Official API if you have access
• Use a dedicated TikTok scraping service

Engagement metrics not loading

TikTok loads engagement data asynchronously. If ScrapeForge captures the page before data loads, try:

• Adding a delay parameter if available
• Using DeepDive which waits for content to render
• Running the request multiple times

Rate limiting on repeated requests

Don't scrape the same TikTok profile more than once per hour. Use caching:

import time

class CachedAnalyzer(TikTokCompetitorAnalyzer):
    def __init__(self, api_key, cache_ttl=3600):
        super().__init__(api_key)
        self.cache = {}
        self.cache_ttl = cache_ttl
    
    def get_profile(self, username):
        cached = self.cache.get(username)
        if cached and time.time() - cached["timestamp"] < self.cache_ttl:
            return cached["data"]
        
        data = super().get_profile(username)
        self.cache[username] = {"data": data, "timestamp": time.time()}
        return data

Next Steps

• Start analyzing competitors: Sign up at searchhive.dev with 500 free credits and extract your first TikTok competitor profile.
• Build your monitoring pipeline: Use the code above to track follower growth and content themes over time.
• Explore the docs: Check searchhive.dev/docs for the full ScrapeForge and DeepDive API reference.

---

Related: /blog/building-ai-agents-with-web-scraping-apis | /compare/firecrawl | /compare/scrapingbee