How to Get a Website's Traffic Sources

You need to see where a website’s traffic really comes from so you can take action: block referral spam in your ad stack, prioritize partners that actually send quality visits, and enrich signups with top countries and engagement. By the end of this guide, you’ll call Klazify’s traffic intelligence endpoint, extract traffic_sources and related fields, and wire them into your pipelines with caching and safeguards.

Why Klazify for website traffic-source intelligence

When your goal is to understand a domain’s traffic mix and act on it at scale—whether for ad placement audits, lead enrichment, or content filtering—classification accuracy and coverage directly impact downstream decisions. Here’s how Klazify helps specifically with traffic-source analysis and adjacent tasks:

• Accurate website categorization using AI: Traffic sources make more sense when paired with category context. Klazify’s content-driven categorization (IAB-mapped) lets you interpret source mixes differently for, say, news portals vs. ecommerce vs. niche blogs.
• Global coverage: Traffic composition and top countries vary by language and region. Klazify evaluates domains in multiple languages, which is key for internationally distributed audiences and country-level enrichment.
• Real-time classification: Traffic patterns move fast. Klazify prioritizes up-to-date content and signals so your analytics, filters, and alerts respond to recent shifts rather than stale datasets.
• Industry-level categories (IAB taxonomy): You can condition business rules on both traffic_sources and category, enabling tactics such as “tighten brand-safety controls on categories with a high social share” or “flag paid-search-heavy domains in sensitive verticals.”
• Simple API integration: A straightforward REST API means you can drop traffic intelligence into ETL jobs, serverless functions, or your ad verification pipeline without bespoke infrastructure.
• Compliance and filtering controls: Traffic-source insight complements brand-safety classification—use both to enforce where ads run, gate content, and score domains for internal policies.

All of this sits alongside company, categorization, logo, and domain signals in the same product, so you can reduce moving parts versus stitching multiple data vendors. Explore the platform at klazify.com or jump straight to a free trial via the link at the end of this post.

What you’ll build

You’ll implement a small service that accepts a domain or URL, fetches its traffic intelligence from Klazify, and returns a normalized object that your systems can consume. Typical actions include:

• Ad tech and brand safety: Audit partners and placements by comparing traffic_sources ratios; pause sources that skew to “referrals” you don’t recognize.
• Content filtering: Combine category with engagement and top_countries to score domains for policy enforcement.
• CRM and lead enrichment: Append monthly_visits and top_countries to company records to prioritize outreach by market presence.
• Analytics: Trend global_rank and country_rank to catch growth or drop-offs, and alert when direct or search contributions shift beyond a control band.

Call the traffic intelligence endpoint

Use the domain traffic intelligence endpoint to retrieve traffic-source composition and related metrics. Authentication uses a Bearer token.

curl -X POST "https://www.klazify.com/api/domain_traffic_intelligence" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://cbsnews.com"}'

Request body:

• url: The URL or domain you want to analyze.

Notes:

• Use a canonical domain or normalized URL to maximize cache hits on your side.
• Failed or unreachable calls are not billed.

Fields you’ll use from traffic intelligence

The response includes fields designed to plug into ranking, enrichment, and filtering workflows. The following are commonly used:

• global_rank: The site’s overall rank worldwide. Lower is better. Use it as a coarse popularity indicator for QA (e.g., ignore outliers below a threshold).
• country_rank: The site’s rank within a specific country. Pair with top_countries to decide which market’s rank to reference.
• category_rank: Popularity within a site’s content category. Useful to compare like-for-like domains regardless of global size.
• monthly_visits: Estimated monthly visits. Cache and track as a trend to detect growth/decline.
• engagement: Engagement metrics such as visit duration or pages per visit. Use to prioritize partnerships or score leads.
• traffic_sources: Breakdown by source types (e.g., direct, search, social, referrals). Normalize to percentages for dashboards and rules like “flag social > 50% for sensitive categories.”
• top_countries: Countries contributing most traffic. Use to enrich CRM with geo presence and route leads to the right region.

Practical handling tips:

• Units: Treat monthly_visits as counts and ranks as integers. When computing deltas, smooth over weekly or monthly windows to reduce noise.
• Time: If you sample domains periodically, record a timestamp when you fetched the metrics and align downstream reporting windows to that timestamp.
• Unknowns: If a field is missing for a niche or new site, fall back to defaults (e.g., traffic_sources = {}), and queue it for recheck later.

Python code to fetch traffic sources and normalize output

import os
import time
import json
import requests
from typing import Any, Dict, List, Tuple

API_KEY = os.getenv("KLAZIFY_API_KEY", "YOUR_API_KEY")
ENDPOINT = "https://www.klazify.com/api/domain_traffic_intelligence"
SESSION = requests.Session()
SESSION.headers.update({
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
"Accept": "application/json"
})

def fetch_traffic_intel(url: str, timeout: int = 20) -> Dict[str, Any]:
payload = {"url": url}
resp = SESSION.post(ENDPOINT, json=payload, timeout=timeout)
# You can inspect HTTP 4xx/5xx here and implement retries/backoff for transient issues.
resp.raise_for_status()
return resp.json()

def normalize_sources(traffic_sources: Dict[str, Any]) -> Dict[str, float]:
# Convert raw numbers to percentages if needed; here we assume they are numeric shares or counts.
values = [v for v in traffic_sources.values() if isinstance(v, (int, float))]
total = float(sum(values)) if values else 0.0
if total == 0:
return {}
return {k: (float(v) / total) for k, v in traffic_sources.items() if isinstance(v, (int, float))}

def select_primary_country(top_countries: List[Dict[str, Any]]) -> Tuple[str, float]:
# Pick the country with the largest share/visits; fall back when unknown.
if not top_countries:
return ("", 0.0)
# Expect items like {"country":"US","share":0.35} or {"country":"US","visits":...}
def score(item: Dict[str, Any]) -> float:
for key in ("share", "visits", "percentage"):
if isinstance(item.get(key), (int, float)):
return float(item[key])
return 0.0
top = max(top_countries, key=score)
return (str(top.get("country", "")), score(top))

def enrich(url: str) -> Dict[str, Any]:
data = fetch_traffic_intel(url)
# Defensive reads in case some fields are missing for new domains.
global_rank = data.get("global_rank")
country_rank = data.get("country_rank")
category_rank = data.get("category_rank")
monthly_visits = data.get("monthly_visits")
engagement = data.get("engagement", {})
traffic_sources = data.get("traffic_sources", {}) or {}
top_countries = data.get("top_countries", []) or []

normalized_sources = normalize_sources(traffic_sources)
primary_country, primary_country_score = select_primary_country(top_countries)

result = {
"domain": url,
"fetched_at": int(time.time()),
"global_rank": global_rank,
"country_rank": country_rank,
"category_rank": category_rank,
"monthly_visits": monthly_visits,
"engagement": engagement,
"traffic_sources": traffic_sources,
"traffic_sources_normalized": normalized_sources,
"top_countries": top_countries,
"primary_country": primary_country,
"primary_country_score": primary_country_score
}
return result

if __name__ == "__main__":
url = "https://cbsnews.com"
data = enrich(url)
print(json.dumps(data, indent=2))

How to use this in production:

• Batching: Maintain a queue of domains to refresh daily or weekly. Run the function above inside a worker that respects your concurrency goals.
• Caching: Store per-domain responses with a TTL that matches your reporting cadence (e.g., 7–30 days). Only force-refresh when you detect critical changes.
• Error handling: Use exponential backoff on transient timeouts or 5xx responses. Do not retry endlessly; schedule a re-fetch later.
• Persistence: Save both the raw payload and your normalized output so you can reprocess with new business rules without re-calling the API.

Official response example (categorization API)

Klazify’s APIs share consistent design. Alongside traffic intelligence, you can enrich the same domain with categories, company data, logo, and similar domains using the categorization endpoint. Below is the official sample response; use this to understand field shapes and how categorization can complement traffic-source analysis in your pipeline.

{
"domain": {
"categories": [
{
"confidence": 0.92,
"name": "/Computers & Electronics/Consumer Electronics",
"IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
},
{
"confidence": 0.89,
"name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
}
],
"social_media": null,
"logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
},
"success": true,
"objects": {
"company": {
"url": "https://www.apple.com/",
"name": "Apple",
"city": "Cupertino",
"stateCode": "CA",
"countryCode": "US",
"employeesRange": "100K+",
"revenue": 274515000000,
"raised": null,
"tags": [
"E-commerce",
"Consumer Electronics",
"Mobile",
"B2C"
],
"tech": [
"omniture_adobe_analytics",
"atlassian_confluence",
"successfactors",
"apache_apex",
"talend",
"oracle_peoplesoft",
"salesforce",
"stripe",
"dell_boomi_atomsphere",
"gigya",
"sage_50cloud",
"quickbooks",
"webmethods",
"apache_tomcat",
"alteryx",
"tibco_rendezvous",
"atlassian_jira",
"..."
]
}
},
"domain_registration_data": {
"domain_age_date": "1987-02-19",
"domain_age_days_ago": "13026",
"domain_expiration_date": "2030-02-20",
"domain_expiration_days_left": "123"
},
"similar_domains": [
"bestbuy.com",
"icloud.com",
"microsoft.com",
"macrumors.com",
"google.com",
"samsung.com",
"twitter.com",
"hp.com",
"bhphotovideo.com",
"dell.com"
]
}

What to use from this sample:

• domain.categories and IAB mapping: Join traffic_sources with categories to build rules (e.g., set different thresholds for social share depending on IAB category).
• objects.company: Use company name and tags to enrich CRM alongside monthly_visits and top_countries from traffic intelligence.
• domain.logo_url: Display a logo in internal tools or partner reviews next to traffic-source charts.
• similar_domains: Expand audits by pulling traffic intelligence for close neighbors.

For reference implementations of all endpoints, see the Documentation.

Operational guidance for scale

Caching and refresh strategy

• Per-domain cache keys: Normalize inputs (lowercase host, strip tracking params) so repeated calls hit your cache.
• TTL: Choose a TTL matching business needs—many teams refresh every 7–30 days and trigger out-of-band refresh on anomalies (e.g., sudden visit spikes).
• Change detection: Store last and current payloads; compute percentage changes for monthly_visits and shares in traffic_sources to drive alerts.

Batching and rate considerations

• Batch domains by priority: High-spend partners, ad placements, or newly seen signups first; long tail later.
• Concurrency: Keep a modest worker pool and add jitter between calls. If you observe throttling, reduce QPS and spread jobs through the day.
• Retries: Only retry on transient failures (timeouts, certain 5xx). Respect backoff and cap retry attempts to avoid storms.

Handling new or low-signal domains

• Missing fields: If traffic_sources or engagement is incomplete, store nulls and reschedule a fetch after a delay.
• Fallbacks: Use categorization and company data as context even when traffic metrics are sparse.
• Human review queues: For critical decisions, route first-seen domains with low confidence to review before allowing ad spend.

Mapping to your own taxonomy

• Use IAB mappings: Klazify returns IAB-aligned categories, which usually map cleanly to ad tech and brand-safety schemes.
• Create a crosswalk table: Maintain a translation from IAB categories to your internal buckets and persist it in your data warehouse for joins.
• Mixed-content sites: If you classify at the URL level using the categorize endpoint, store both domain-level and URL-level categories to refine decisions.

Putting it to work: examples of actions

• Block referral spam: If a partner’s placements send traffic to domains where referrals dominate but top referrers don’t match your allowlist, quarantine the source.
• Prioritize publishers: For news or entertainment categories, prefer domains where search and direct dominate over low-quality social bursts.
• Geo-routing leads: If top_countries shows strong presence in a particular market, assign the account to that region’s sales team and localize outreach.
• Budget reallocation: Monitor monthly_visits and global_rank to shift spend toward rising domains and away from declining ones.

Data model and storage pattern

Consider a fact table keyed by domain and fetched_at with these columns:

• domain (string), fetched_at (timestamp)
• global_rank (int), country_rank (int), category_rank (int)
• monthly_visits (numeric)
• engagement (JSON/STRUCT)
• traffic_sources (JSON/STRUCT), traffic_sources_normalized (JSON/STRUCT)
• top_countries (ARRAY of STRUCT)
• category (string), iab_code (string), company_name (string), logo_url (string)

Downstream, build materialized views to compute rolling averages and anomaly scores. Persist the raw JSON alongside extracted fields so you can reprocess with new rules without calling the API again.

Security, monitoring, and QA

• Secrets: Store YOUR_API_KEY in a vault or managed secret store; never hardcode.
• Observability: Log request IDs, domains, status codes, and latency. Alert on sustained error rates.
• Validation: Add schema guards to ensure expected keys exist before use; default to safe behavior when data is missing.
• Cost control: Failed or unreachable calls are not billed. Use caching and batching to minimize redundant calls. The Starter plan is $39.99/mo with a 7-day trial.

Quick reference: endpoints you’ll use

• Main traffic endpoint: POST https://www.klazify.com/api/domain_traffic_intelligence
• Classification endpoint (for categories, company, logo, and more): https://www.klazify.com/api/categorize

Explore more features, including social URL detection and similar domains, on the Klazify homepage and in the MCP.

FAQ

• How often should I refresh traffic intelligence? For most pipelines, a 7–30 day cadence works. Refresh more frequently for high-spend partners or when you detect anomalies in source shares or visits.
• What if a field like traffic_sources is missing? Store nulls, apply conservative defaults in your rules, and schedule a re-fetch. Also combine with categorization and company context for interim decisions.
• Can I map Klazify categories to my internal taxonomy? Yes. Use the IAB-mapped category values from the categorize endpoint, maintain a crosswalk table, and join it to your traffic data by domain.
• How do I avoid hitting rate limits? Batch domains, add jitter between requests, and cache responses with a TTL. If you see throttling, reduce concurrency and spread jobs throughout the day.
• Do failed or unreachable calls count toward usage? No. Failed or unreachable calls are not billed.

Ready to pull a domain’s traffic sources and enrich your decisions? Create your free account and start integrating today: Register. For full parameter and schema details, check the Documentation.