How to Get a Website's Category in Python

November 22, 2026
How to Get a Website's Category in Python

You’re building a Python task that needs a website’s category to make a decision in real time: block “News” on a school network, enrich a CRM signup with industry tags, or audit where your ads run. In this guide you’ll call a single API, parse the response in Python, and wire the category into your pipeline with caching, fallback logic, and clear handling for unknown domains.

Why Klazify is a strong fit for classifying websites from Python

When your goal is to get a website’s category inside a Python workflow, you need accurate labels, low latency, and a response that’s easy to integrate. Here’s how Klazify maps to that need set:

Illustration: How to Get a Website's Category in Python
  • Accurate website categorization using AI: The API analyzes real website content rather than only metadata. That matters for Python tasks like content filtering or contextual targeting where you’ll make automated decisions off the category string.
  • Global coverage: You’ll encounter multilingual sites in production pipelines. Klazify’s approach is designed to handle global content, so your Python service can categorize domains your users encounter worldwide.
  • Real-time classification: Instead of relying on stale datasets, the endpoint performs up-to-date analysis, which helps your classifier react to content changes without manual re-ingestion flows.
  • IAB taxonomy mapping: You get industry-aligned categories suitable for ad tech, brand safety, and analytics. This makes it straightforward to align Python business rules to IAB categories.
  • Simple API integration: A clean REST endpoint with a JSON body means a few lines of Python are enough to fetch categories, tech stack hints, logos, and company data in one call.
  • Compliance and filtering: If your Python service enforces allow/block policies, the categorical labels are granular enough to let you whitelist or exclude specific topics with predictable logic.

If your use case is a school network filter, a brand-safety control for ad placements, or signup enrichment in a Flask or FastAPI backend, Klazify’s single-call design keeps your Python code compact while returning all the fields necessary to decide, store, and audit.

What you’ll build: a Python categorizer you can drop into any pipeline

We’ll cover how to:

  • Call the categorization endpoint with a URL and a Bearer token.
  • Extract the top category, confidence, logo, and company details.
  • Cache results per domain to save time and calls.
  • Handle unknown or new domains safely.
  • Map IAB-inspired category strings to your own taxonomy.

To follow along, create a free account and generate an API key. Starter is $39.99/mo with a 7-day trial, and failed or unreachable calls are not billed.

Endpoint and authentication

Klazify’s categorization endpoint accepts a JSON payload with the URL you want to classify. Authenticate using a Bearer token in the Authorization header.

  • Endpoint: https://www.klazify.com/api/categorize
  • Method: POST
  • Headers: Authorization: Bearer YOUR_API_KEY and Content-Type: application/json
  • Body: {"url":"the website or page you want to classify"}

Sign up for an API key here: Register. For full parameter details, see the Documentation.

Quick test with curl (copy/paste)

This is a minimal request you can run in your terminal. It posts a URL to the categorizer and returns JSON with category data, logos, social links, company information, technology stack, and more.

curl -X POST "https://www.klazify.com/api/categorize" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://cbsnews.com"}'

From this call, you’ll get a top category for the page or domain. For example, a news site will be categorized as /News along with a confidence score like 0.97. You’ll also receive fields you can use to enrich your records (logo_url, company details, and more).

Python example: get category, confidence, and enrichment fields

The snippet below shows a production-ready pattern: request with timeouts, parse expected fields, and implement a basic in-memory cache keyed by the normalized domain.

import os
import time
import json
import hashlib
import requests
from urllib.parse import urlparse

API_URL = "https://www.klazify.com/api/categorize"
API_KEY = os.getenv("KLAZIFY_API_KEY", "YOUR_API_KEY")

# Simple in-memory cache: dict of {domain: (timestamp, response_json)}
CACHE = {}
CACHE_TTL_SECONDS = 60 * 60 * 24 * 7  # cache for 7 days

def normalize_domain(url):
    parsed = urlparse(url if url.startswith("http") else f"https://{url}")
    host = parsed.netloc.lower()
    # strip common "www." prefix for consistency
    if host.startswith("www."):
        host = host[4:]
    return host

def get_cached_or_fetch(url):
    domain = normalize_domain(url)
    now = time.time()
    if domain in CACHE:
        ts, payload = CACHE[domain]
        if now - ts < CACHE_TTL_SECONDS:
            return payload

    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    }
    payload = {"url": url}

    # basic retry for transient errors
    backoffs = [0.5, 1.0, 2.0]
    last_err = None
    for delay in backoffs:
        try:
            resp = requests.post(API_URL, headers=headers, json=payload, timeout=10)
            if resp.status_code == 200:
                data = resp.json()
                # Cache only successful responses
                CACHE[domain] = (now, data)
                return data
            elif resp.status_code in (429, 500, 502, 503, 504):
                time.sleep(delay)
                continue
            else:
                # Non-retryable status; return parsed error for caller to decide
                return {"success": False, "status": resp.status_code, "text": resp.text}
        except requests.RequestException as e:
            last_err = str(e)
            time.sleep(delay)
    return {"success": False, "error": last_err or "request_failed"}

def extract_category_info(data):
    # Defensive parsing: fields may be missing or null
    domain = data.get("domain") or {}
    categories = domain.get("categories") or []
    top_cat = categories[0] if categories else {}
    cat_name = top_cat.get("name")  # e.g., "/News" or "/Computers & Electronics/Consumer Electronics"
    confidence = top_cat.get("confidence")
    logo_url = domain.get("logo_url")
    company = (data.get("objects") or {}).get("company") or {}
    company_name = company.get("name")
    tech = company.get("tech") or []
    return {
        "category": cat_name,
        "confidence": confidence,
        "logo_url": logo_url,
        "company_name": company_name,
        "tech": tech,
        "raw": data,
    }

if __name__ == "__main__":
    test_url = "https://cbsnews.com"
    result = get_cached_or_fetch(test_url)
    info = extract_category_info(result)

    print("Category:", info["category"])
    print("Confidence:", info["confidence"])
    print("Company:", info["company_name"])
    print("Logo URL:", info["logo_url"])
    print("Has Tech Stack:", len(info["tech"]) > 0)

In your own pipeline, branch on info["category"] to block, allow, or enrich records. For example, if your policy is to exclude “/News” from ad placements, check for category prefixes like “/News” and confidence above a threshold you choose.

Example response and how to use it

Below is an official example of the response format. You’ll see a domain.categories array with a confidence score, plus logo, social, company, registration data, and similar domains you can use for enrichment and analytics.


{
    "domain": {
        "categories": [
            {
                "confidence": 0.92,
                "name": "/Computers & Electronics/Consumer Electronics",
                "IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
            },
            {
                "confidence": 0.89,
                "name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
            }
        ],
        "social_media": null,
        "logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
    },
    "success": true,
    "objects": {
        "company": {
            "url": "https://www.apple.com/",
            "name": "Apple",
            "city": "Cupertino",
            "stateCode": "CA",
            "countryCode": "US",
            "employeesRange": "100K+",
            "revenue": 274515000000,
            "raised": null,
            "tags": [
                "E-commerce",
                "Consumer Electronics",
                "Mobile",
                "B2C"
            ],
            "tech": [
                "omniture_adobe_analytics",
                "atlassian_confluence",
                "successfactors",
                "apache_apex",
                "talend",
                "oracle_peoplesoft",
                "salesforce",
                "stripe",
                "dell_boomi_atomsphere",
                "gigya",
                "sage_50cloud",
                "quickbooks",
                "webmethods",
                "apache_tomcat",
                "alteryx",
                "tibco_rendezvous",
                "atlassian_jira",
                "..."
            ]
        }
    },
    "domain_registration_data": {
        "domain_age_date": "1987-02-19",
        "domain_age_days_ago": "13026",
        "domain_expiration_date": "2030-02-20",
        "domain_expiration_days_left": "123"
    },
    "similar_domains": [
        "bestbuy.com",
        "icloud.com",
        "microsoft.com",
        "macrumors.com",
        "google.com",
        "samsung.com",
        "twitter.com",
        "hp.com",
        "bhphotovideo.com",
        "dell.com"
    ]
}

How to use the fields:

  • domain.categories: An ordered list of categories with confidence. Use categories[0].name as your primary label and categories[0].confidence as your filter threshold.
  • domain.logo_url: Store or display this when building dashboards or CRM views.
  • objects.company: Enrichment such as company name, location, employeesRange, revenue, tags, and tech stack. Ideal for lead scoring or segmentation.
  • domain_registration_data: Helpful for fraud checks or risk scoring (e.g., very young domains may require review).
  • similar_domains: Expand prospecting lists or perform competitive analysis in your analytics jobs.

Turning categories into decisions

Here are common actions you can implement immediately after parsing the JSON in Python:

  • Content filtering: Blocklist categories by prefix (e.g., “/News”). If using a confidence threshold, set a minimum like 0.8 and add a manual review queue for borderline cases.
  • Ad placement controls: Exclude categories from specific campaigns. Log the returned category next to each impression or placement for auditing.
  • Signup enrichment: On form submit, hit the API with the user’s company domain, then attach objects.company.tags and domain.categories[0].name to the lead record.
  • Analytics & segmentation: Group domains by their top-level category path segment for high-level dashboards, then drill into subcategories for precise reporting.

Mapping IAB-style categories to your own taxonomy

Many teams map Klazify’s category path to an internal taxonomy. A simple approach is to match path prefixes to your canonical labels. Example rules:

  • If name starts with “/News”, map to internal: “News & Journalism”.
  • If name starts with “/Computers & Electronics”, map to internal: “Technology”.
  • Else, keep the full path as-is and flag for later review.

If you already maintain category mappings, store them in a dictionary keyed by prefix, and apply them after extracting domain.categories[0].name. For organizational governance, keep a changelog of mapping revisions and the date you deployed them.

Operational details that save time in production

Batching and concurrency

If you’re classifying large lists of domains, parallelize requests with a worker pool and de-duplicate inputs across jobs. Normalize domains (strip “www.” and enforce lowercase) so identical hosts hit cache rather than the network.

Rate limiting and retries

Build exponential backoff on 429 or 5xx responses and log retry attempts. Avoid tight infinite loops; cap retries and surface partial results to keep pipelines flowing. Inspect response codes for non-retryable errors and fail fast for those cases.

Caching strategy

Categories are relatively stable but can change. Cache a domain’s response for 7 to 30 days depending on your tolerance for drift, then refresh asynchronously. In Python, use an in-memory cache for small services or Redis for distributed systems. Cache keys should be normalized domains. Store the raw JSON so you don’t have to re-fetch when you later decide to use more fields.

Handling unknown or new domains

Sometimes a domain is new or temporarily unreachable. Design your parser to handle missing or null fields gracefully. If categories are absent, fall back to a default route (e.g., “Unknown”) and enqueue a recheck job. Because failed or unreachable calls are not billed, scheduled retries are cost-safe.

Data governance and auditing

Persist the category string, confidence, and the timestamp you obtained it. When decisions are downstream (e.g., blocking or ad exclusion), keep these fields alongside your decision logs so you can audit exactly why a choice was made.

How different fields support different workflows

Field What it tells you Common uses Notes
domain.categories[0].name Primary category path Block/allow lists, ad safety, segmentation Match by prefix for broad rules
domain.categories[0].confidence Strength of classification Thresholding, manual review routing Adjust per use case
domain.logo_url Logo image URL CRM enrichment, dashboards Cache for UI speed
objects.company Firmographic data Lead scoring, ABM, targeting Combine with categories
domain_registration_data Age and expiration info Risk scoring, fraud checks Useful for new domains
similar_domains Related sites Prospecting, competitive sets Feed into next crawl

Integrating into your Python services

  • Web backends (Flask/FastAPI): On form submits with a company domain, call the API in a background task and update the record when done. Show a “category pending” tag until enrichment completes.
  • ETL pipelines (Airflow, Prefect): Normalize input domains, call the API with concurrency controls, and write enriched results to your warehouse with the raw JSON in a VARIANT/JSON column for flexibility.
  • Streaming (Kafka consumers): Add a caching layer, dedupe per domain, and queue reclassifications on a schedule rather than every event.

For service-to-service policies, track both the category string and a boolean derived from your rule set (e.g., is_blocked) so downstream services don’t need to re-implement classification logic.

Testing and observability

  • Unit tests: Mock the API response and validate your mapping logic from category paths to decisions.
  • Contract tests: Ensure your JSON parser handles absent fields (e.g., social_media: null) without exceptions.
  • Metrics: Track request counts, success/failed responses, average latency, cache hit ratio, and distribution of categories.
  • Alerts: Notify on spikes in unknown classifications or retry rates, which can indicate input issues or connectivity problems.

Governance and access control

To administer keys, track usage sources, and manage data access across environments, see the MCP page. Keep separate keys for dev, staging, and prod, and rotate them on a schedule consistent with your security policies.

Budgeting and rollout tips

  • Plan your cache TTL and concurrency to keep latency and spend predictable.
  • Use sampling in analytics pipelines to test new rules before full rollout.
  • Remember: failed or unreachable calls are not billed, so you can implement safe retries without incurring extra charges for failures.
  • Starter plan is $39.99/month with a 7-day trial, which is typically enough to prototype and validate your integration.

Putting it to work: end-to-end flow

  1. Normalize inputs: Lowercase, strip “www.”, and ensure you send full URLs where page-level classification matters.
  2. Call the endpoint with your Bearer token and JSON body.
  3. Parse domain.categories. If empty, record as “Unknown” and enqueue a retry.
  4. Apply business rules: category prefix checks, confidence threshold, and custom mapping to your taxonomy.
  5. Enrich records with logo_url, objects.company.* and store raw JSON for auditability.
  6. Cache results by domain; schedule periodic refreshes to catch meaningful content changes.
  7. Log each decision alongside the inputs (url, category, confidence, timestamp).

FAQ

  • Can I classify a specific page URL instead of just the root domain?
    Yes. Send the full page URL in the {"url": "..."} payload to classify content at the page level when needed.
  • How should I pick a confidence threshold?
    Start with a conservative minimum (e.g., 0.8), sample borderline cases, and adjust based on false-positive/false-negative tolerance for your use case.
  • What taxonomy do the categories follow?
    Categories map to the IAB taxonomy, which works well for ad tech, brand safety, and analytics use cases.
  • What happens if a target site is down when I call the API?
    Handle null or missing fields gracefully, retry with backoff, and note that failed or unreachable calls are not billed.
  • How often should I refresh cached results?
    Many teams refresh every 7–30 days. Pick a TTL that balances change detection against request volume and latency needs.

Build your Python categorizer in minutes and plug it into filtering, enrichment, or analytics. Create your key now and ship your integration: Register. For full endpoint details and field descriptions, visit the Documentation.

Ready to use Klazify?

Start classifying websites, enriching company data, and exploring web intelligence.

Get Started Free