How to Get a Website's Traffic Sources in Python
You need to see where a website’s traffic comes from and turn that into a signal you can use in code: to audit acquisition mix, score leads, or prioritize outreach. By the end of this guide, you’ll fetch a domain’s traffic sources in Python, extract engagement and top-country data, and wire the results into your pipeline with caching, batching, and fallback strategies.
Why Klazify is the best fit for traffic-source intelligence
When your goal is to retrieve traffic sources programmatically and use them to enrich domains at scale, accuracy, coverage, and easy integration matter. Klazify’s domain intelligence API is built for this exact workflow, and it complements traffic-source data with categories, ranks, and company context so you can automate decisions, not just log charts.
- Accurate website categorization using AI: Klazify analyzes real site content and structure, not just metadata. For traffic-source analysis, this matters because understanding what a site is about helps interpret the acquisition mix (e.g., high social share for entertainment vs. high search share for technical docs).
- Global coverage: You’ll get traffic intel across multiple languages and regions, which pairs well with country-level breakdowns to support regional targeting and brand safety.
- Real-time classification: Fresh categorization and updated domain data ensure your downstream actions (allowlists, enrichment, attribution rules) don’t drift.
- Industry-level categories (IAB): Align traffic patterns with IAB taxonomy for ad tech, contextual targeting, or brand safety logic.
- Simple API integration: A clean REST endpoint for traffic intelligence plus categorization, social, logos, and company data from the same platform means fewer moving parts. See the Documentation for parameters and response fields.
- Compliance and filtering: Use categories and traffic mix together to flag domains for deeper review (e.g., unexpected direct spikes) or to whitelist known, stable sources.
If you’re building analytics, ad verification, cybersecurity, or enrichment features, Klazify lets you join traffic-source signals to standardized categories and company context with one integration. Create a free account to try it: Register.
What you’ll build: a Python workflow for traffic sources
We’ll use Klazify’s traffic intelligence endpoint to:
- POST a domain and retrieve traffic_sources, top_countries, engagement, and rank fields.
- Parse and normalize the response for downstream use (e.g., converting shares to percentages, selecting top-N sources).
- Cache per domain to avoid re-fetching unchanged results.
- Batch multiple domains safely to respect rate limits and reduce latency.
We’ll also show how to combine traffic intelligence with categorization to improve scoring and filtering.
Endpoint and authentication
To retrieve traffic intelligence, send a POST request to the following endpoint with a JSON body containing a url. Authenticate with a Bearer token. Failed or unreachable calls are not billed.
- Endpoint: https://www.klazify.com/api/domain_traffic_intelligence
- Method: POST
- Headers: Authorization: Bearer YOUR_API_KEY, Content-Type: application/json
- Body: {"url":"https://example.com"}
Starter is $39.99/mo with a 7-day trial. For full parameter details and returned fields, consult the Documentation.
Copy-pasteable cURL request
This example shows a request to the traffic intelligence endpoint. The domain appears here only for demonstration.
curl -X POST "https://www.klazify.com/api/domain_traffic_intelligence" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://cbsnews.com"}'
On success, the response includes ranks, monthly visits, engagement metrics, traffic_sources, and top_countries. We’ll parse those fields in Python next.
Python code: fetch and use traffic sources
The snippet below demonstrates how to call the endpoint, handle timeouts, and extract the fields you’ll likely use in data pipelines.
import os
import json
import time
import requests
API_KEY = os.getenv("KLAZIFY_API_KEY", "YOUR_API_KEY")
ENDPOINT = "https://www.klazify.com/api/domain_traffic_intelligence"
def fetch_traffic_intel(url, timeout=15):
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
payload = {"url": url}
resp = requests.post(ENDPOINT, headers=headers, json=payload, timeout=timeout)
# Handle transient errors with a lightweight retry (exponential backoff)
if resp.status_code >= 500:
time.sleep(1.0)
resp = requests.post(ENDPOINT, headers=headers, json=payload, timeout=timeout)
resp.raise_for_status()
return resp.json()
def normalize_traffic_sources(ts):
# Expect a structure with source keys and numeric shares or counts.
# Convert to a sorted list of (source, value) pairs for stable processing.
if not isinstance(ts, dict):
return []
items = [(k, v) for k, v in ts.items() if isinstance(v, (int, float))]
return sorted(items, key=lambda x: x[1], reverse=True)
def top_countries_list(tc, topn=5):
# Expect a list or dict-like structure. Keep a defensive parsing stance.
if isinstance(tc, list):
# Already a list of country entries; keep top N
return tc[:topn]
if isinstance(tc, dict):
# Convert to sorted list by numeric value if present
items = [(k, v) for k, v in tc.items() if isinstance(v, (int, float))]
return [{"country": k, "value": v} for k, v in sorted(items, key=lambda x: x[1], reverse=True)[:topn]]
return []
def summarize_domain(url):
data = fetch_traffic_intel(url)
# Defensive access: only read fields if present
global_rank = data.get("global_rank")
country_rank = data.get("country_rank")
category_rank = data.get("category_rank")
monthly_visits = data.get("monthly_visits")
engagement = data.get("engagement") # e.g., visit_duration, pages_per_visit, bounce_rate
traffic_sources = data.get("traffic_sources")
top_countries = data.get("top_countries")
# Normalize structures for downstream logic
ts_sorted = normalize_traffic_sources(traffic_sources)
top_c = top_countries_list(top_countries, topn=5)
return {
"domain": url,
"global_rank": global_rank,
"country_rank": country_rank,
"category_rank": category_rank,
"monthly_visits": monthly_visits,
"engagement": engagement,
"traffic_sources_sorted": ts_sorted,
"top_countries_top5": top_c
}
if __name__ == "__main__":
domain = "https://example.com"
result = summarize_domain(domain)
print(json.dumps(result, indent=2))
How to use the fields:
- global_rank, country_rank, category_rank: Prioritize domains by reach; combine with category to segment influencers or partners.
- monthly_visits: A straightforward scale input for scoring, thresholds, or capacity planning.
- engagement: Pair pages_per_visit and visit_duration signals with traffic sources to understand quality by acquisition type.
- traffic_sources: Drive mix-based rules. Example: if search share is low for a documentation site, flag it for SEO review; if referrals are unexpectedly high, vet the sources.
- top_countries: Localize outreach, bids, or moderation coverage based on where traffic concentrates.
Join traffic intel with categorization and company data
The traffic intelligence results become more actionable when combined with categorization. For example, you might:
- Require a minimum global_rank threshold for certain IAB categories before recommending a partnership.
- Allowlist domains with stable engagement in your target category and discount sources that show volatile referral swings.
Use the main categorization endpoint to retrieve categories, IAB mappings, logo URLs, and company information. Here is the official example response format (include it in your integration tests as a fixture so your parser is robust to nested fields):
{
"domain": {
"categories": [
{
"confidence": 0.92,
"name": "/Computers & Electronics/Consumer Electronics",
"IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
},
{
"confidence": 0.89,
"name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
}
],
"social_media": null,
"logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
},
"success": true,
"objects": {
"company": {
"url": "https://www.apple.com/",
"name": "Apple",
"city": "Cupertino",
"stateCode": "CA",
"countryCode": "US",
"employeesRange": "100K+",
"revenue": 274515000000,
"raised": null,
"tags": [
"E-commerce",
"Consumer Electronics",
"Mobile",
"B2C"
],
"tech": [
"omniture_adobe_analytics",
"atlassian_confluence",
"successfactors",
"apache_apex",
"talend",
"oracle_peoplesoft",
"salesforce",
"stripe",
"dell_boomi_atomsphere",
"gigya",
"sage_50cloud",
"quickbooks",
"webmethods",
"apache_tomcat",
"alteryx",
"tibco_rendezvous",
"atlassian_jira",
"..."
]
}
},
"domain_registration_data": {
"domain_age_date": "1987-02-19",
"domain_age_days_ago": "13026",
"domain_expiration_date": "2030-02-20",
"domain_expiration_days_left": "123"
},
"similar_domains": [
"bestbuy.com",
"icloud.com",
"microsoft.com",
"macrumors.com",
"google.com",
"samsung.com",
"twitter.com",
"hp.com",
"bhphotovideo.com",
"dell.com"
]
}
Fields you’ll likely use from this response:
- domain.categories[].name and domain.categories[].confidence: Attach IAB-aligned categories to traffic-source records so downstream users can filter by context.
- domain.logo_url: Helpful for internal dashboards where you visualize acquisition mix per domain.
- objects.company.*: Enrich CRM records with company location, size ranges, and tags.
- domain_registration_data.*: Keep an eye on soon-to-expire domains when traffic anomalies arise.
- similar_domains: Expand discovery and run the same traffic-intelligence workflow on peer sites.
JSON response examples you can test against
Include these in your unit tests to verify parsing stability for nested arrays, nulls, and large lists. They are identical to ensure you always test with the official structure before layering your own mocks.
Sample A: Categorization
{
"domain": {
"categories": [
{
"confidence": 0.92,
"name": "/Computers & Electronics/Consumer Electronics",
"IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
},
{
"confidence": 0.89,
"name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
}
],
"social_media": null,
"logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
},
"success": true,
"objects": {
"company": {
"url": "https://www.apple.com/",
"name": "Apple",
"city": "Cupertino",
"stateCode": "CA",
"countryCode": "US",
"employeesRange": "100K+",
"revenue": 274515000000,
"raised": null,
"tags": [
"E-commerce",
"Consumer Electronics",
"Mobile",
"B2C"
],
"tech": [
"omniture_adobe_analytics",
"atlassian_confluence",
"successfactors",
"apache_apex",
"talend",
"oracle_peoplesoft",
"salesforce",
"stripe",
"dell_boomi_atomsphere",
"gigya",
"sage_50cloud",
"quickbooks",
"webmethods",
"apache_tomcat",
"alteryx",
"tibco_rendezvous",
"atlassian_jira",
"..."
]
}
},
"domain_registration_data": {
"domain_age_date": "1987-02-19",
"domain_age_days_ago": "13026",
"domain_expiration_date": "2030-02-20",
"domain_expiration_days_left": "123"
},
"similar_domains": [
"bestbuy.com",
"icloud.com",
"microsoft.com",
"macrumors.com",
"google.com",
"samsung.com",
"twitter.com",
"hp.com",
"bhphotovideo.com",
"dell.com"
]
}
Sample B: Categorization (duplicate for testing)
{
"domain": {
"categories": [
{
"confidence": 0.92,
"name": "/Computers & Electronics/Consumer Electronics",
"IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
},
{
"confidence": 0.89,
"name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
}
],
"social_media": null,
"logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
},
"success": true,
"objects": {
"company": {
"url": "https://www.apple.com/",
"name": "Apple",
"city": "Cupertino",
"stateCode": "CA",
"countryCode": "US",
"employeesRange": "100K+",
"revenue": 274515000000,
"raised": null,
"tags": [
"E-commerce",
"Consumer Electronics",
"Mobile",
"B2C"
],
"tech": [
"omniture_adobe_analytics",
"atlassian_confluence",
"successfactors",
"apache_apex",
"talend",
"oracle_peoplesoft",
"salesforce",
"stripe",
"dell_boomi_atomsphere",
"gigya",
"sage_50cloud",
"quickbooks",
"webmethods",
"apache_tomcat",
"alteryx",
"tibco_rendezvous",
"atlassian_jira",
"..."
]
}
},
"domain_registration_data": {
"domain_age_date": "1987-02-19",
"domain_age_days_ago": "13026",
"domain_expiration_date": "2030-02-20",
"domain_expiration_days_left": "123"
},
"similar_domains": [
"bestbuy.com",
"icloud.com",
"microsoft.com",
"macrumors.com",
"google.com",
"samsung.com",
"twitter.com",
"hp.com",
"bhphotovideo.com",
"dell.com"
]
}
Sample C: Categorization (duplicate for contract testing)
{
"domain": {
"categories": [
{
"confidence": 0.92,
"name": "/Computers & Electronics/Consumer Electronics",
"IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
},
{
"confidence": 0.89,
"name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
}
],
"social_media": null,
"logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
},
"success": true,
"objects": {
"company": {
"url": "https://www.apple.com/",
"name": "Apple",
"city": "Cupertino",
"stateCode": "CA",
"countryCode": "US",
"employeesRange": "100K+",
"revenue": 274515000000,
"raised": null,
"tags": [
"E-commerce",
"Consumer Electronics",
"Mobile",
"B2C"
],
"tech": [
"omniture_adobe_analytics",
"atlassian_confluence",
"successfactors",
"apache_apex",
"talend",
"oracle_peoplesoft",
"salesforce",
"stripe",
"dell_boomi_atomsphere",
"gigya",
"sage_50cloud",
"quickbooks",
"webmethods",
"apache_tomcat",
"alteryx",
"tibco_rendezvous",
"atlassian_jira",
"..."
]
}
},
"domain_registration_data": {
"domain_age_date": "1987-02-19",
"domain_age_days_ago": "13026",
"domain_expiration_date": "2030-02-20",
"domain_expiration_days_left": "123"
},
"similar_domains": [
"bestbuy.com",
"icloud.com",
"microsoft.com",
"macrumors.com",
"google.com",
"samsung.com",
"twitter.com",
"hp.com",
"bhphotovideo.com",
"dell.com"
]
}
Turn traffic-source data into decisions
With the parsed fields, you can set rules that influence pipelines immediately:
- Enrichment: Write traffic_sources and top_countries to your warehouse next to domain and category. Use dbt or your preferred transform tool to compute mix ratios.
- Lead scoring: Improve scores for domains with high monthly_visits and engagement paired with relevant IAB categories.
- QA and anomaly detection: Alert if direct share spikes while engagement drops, or if top_country flips week-over-week.
- Brand safety: Combine category with known referral patterns to flag unexpected acquisition sources for manual review.
Batching, rate limits, and latency
For high throughput, group domains into small batches in your job queue. Keep concurrency modest and back off on 429/5xx responses. While exact rate limits are documented in your plan, design with:
- Queued jobs: Enqueue domains and let workers pull tasks to control concurrency.
- Exponential backoff: Retry transient failures with jitter and cap the maximum retries.
- Partial progress: Commit processed domains incrementally to avoid rework.
When you rerun enrichment, skip domains present in your cache unless a refresh is required. This improves both latency and cost, and failed or unreachable calls are not billed.
Caching and update strategy
Cache traffic intelligence per domain in your database keyed by the normalized domain (e.g., eTLD+1). Add a fetched_at timestamp, and refresh on a schedule that matches your business cadence. For streaming use cases, add a staleness threshold to re-fetch on read if needed.
- Normalize input: Canonicalize domains to avoid duplicates (e.g., handling www and HTTP/HTTPS variations).
- Schema design: Store global_rank, country_rank, category_rank, monthly_visits, engagement.*, traffic_sources, and top_countries as typed columns or as a JSON blob based on your analytics stack.
- Immutability: Keep historical snapshots for time-series comparisons and anomaly detection.
Handling unknown or new domains
Sometimes a domain has limited historical signals. Build a fallback flow:
- Retry policy: Retry transient network timeouts and 5xx with backoff.
- Graceful defaults: If traffic_sources is missing, store an empty structure and schedule a recheck later.
- Augment with categorization: Even when traffic data is sparse, categories and company context still add value for routing or manual review.
Mapping to your own taxonomy
If your internal taxonomy differs from IAB, maintain a mapping table in your data warehouse. Use domain.categories[].name and, when available, IAB fields (e.g., IAB-632-596) to map Klazify categories to internal labels. Store both raw and mapped categories for traceability. This mapping lets you segment traffic-source mixes by your own product or content types.
Field-to-action reference
| Field | What it tells you | Typical action |
|---|---|---|
| global_rank | Overall reach/importance | Prioritize crawling, partnerships, or QA on high-impact domains |
| country_rank | Dominance in a specific country | Localize messaging and evaluate geo-specific campaigns |
| category_rank | Standing within its content niche | Benchmark against peers; feed “top of category” badges |
| monthly_visits | Scale indicator | Gate features or sales tiers by traffic size |
| engagement | Visit quality (duration, pages, bounce) | Score leads and qualify placements |
| traffic_sources | Mix across direct, search, social, referrals, etc. | Set auditing rules and growth opportunities by source |
| top_countries | Geographic distribution | Route to regional teams; adjust moderation/coverage |
Operational checklist before shipping
- Input validation: Verify url shape and normalize domains.
- Timeouts and retries: Set request timeouts and exponential backoff.
- Caching: Store results per domain; add fetched_at and next_refresh_at timestamps.
- Observability: Log status codes and durations; capture payload sizes.
- Data quality: Alert on missing critical fields; add a dead-letter queue for reprocessing.
- Security: Keep YOUR_API_KEY in a secret store or environment variable.
Governance and MCP
If you maintain internal policies for data access and automation, align your workflows with your governance process. For centralized policy and control, see MCP.
Where to go next
Use the traffic intelligence endpoint alongside classification to build robust enrichment, analytics, and QA. Keep your integration simple: one provider for traffic, categories, logos, social links, and company data reduces drift and maintenance. Explore more at klazify.com and get your key here: Register.
FAQ
-
Which endpoint returns traffic sources?
Use POST https://www.klazify.com/api/domain_traffic_intelligence with Authorization: Bearer YOUR_API_KEY and a JSON body containing {"url":"https://example.com"}. -
Can I combine traffic sources with IAB categories?
Yes. Fetch categorization from the main endpoint and join on domain to segment acquisition mix by category in your warehouse. -
How should I cache results?
Cache per normalized domain with a fetched_at timestamp. Refresh on a cadence that fits your pipeline and re-fetch on read if the result is stale. -
What happens if the request fails or the host is unreachable?
Failed or unreachable calls are not billed. Implement retries with backoff for transient issues. -
Is there a free trial?
Yes, the Starter plan is $39.99/mo with a 7-day trial. Create your account at Register.
Ready to enrich domains with traffic sources, ranks, and categories in one place? Start with the free trial at Register and build your integration using the Documentation. You can always learn more about the platform at klazify.com.
Ready to use Klazify?
Start classifying websites, enriching company data, and exploring web intelligence.
Get Started Free