How to Classify a Website in Real Time

You need to decide what to do with a domain the instant it appears in your system—block it for a brand-safety rule, enrich a signup form, or audit where an ad impression is about to run. By the end of this guide, you’ll have a working, copy‑pasteable integration that classifies any URL in real time using Klazify, then turns the result into an action in your pipeline.

Why Klazify is the right fit for instant website decisions

Real-time decisions fail if the classifier is stale, slow, or too coarse. Klazify’s real-time categorization endpoint analyzes the page content live, making it practical for the moment you need to block, allow, or enrich:

• Accurate classification using AI: Klazify analyzes on-page content (not just metadata), which helps disambiguate ambiguous or multi-topic sites at request time.
• Global coverage: Real sites appear in many languages and regions. Klazify’s models handle worldwide content so your rule set works across markets.
• Real-time classification: Every call is live. You don’t rely on an outdated database when brand-safety or policy decisions are on the line.
• Industry-level categories (IAB): Output aligns to IAB taxonomy, which is standard for ad ops, brand safety, and contextual targeting.
• Simple API integration: A single POST request with JSON in/JSON out slots into typical microservices and serverless setups.
• Compliance and filtering: Granular categories help you confidently block or whitelist sensitive topics, while letting allowed content through.

If your concrete goal is to 1) block a category, 2) enrich a new domain at signup, or 3) audit where ads are about to run, Klazify’s real-time endpoint is built to serve those exact decisions with minimal glue code.

What you’ll build: a real-time categorizer that triggers actions

We’ll integrate the real-time endpoint to:

• Classify a URL on demand via a single HTTP POST request.
• Parse returned categories and confidence scores.
• Map IAB-aligned categories to a policy table (block/allow) or an enrichment schema.
• Cache results by domain to reduce latency and cost on subsequent hits.

We’ll also cover operational details: batching, handling unknown or new domains, and mapping categories to your own taxonomy.

Make a live classification call

Use the dedicated real-time endpoint. Every call analyzes the target at request time. Do not send a refresh flag—no pre-warmed cache is used here.

curl -X POST "https://www.klazify.com/api/real_time_categorization" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://cbsnews.com"}'

Notes:

• Authorization: Bearer YOUR_API_KEY – replace YOUR_API_KEY with your token.
• Body: JSON with a single url field. Use full URLs or domains as they appear in your logs or request stream.
• Billing: Failed or unreachable calls are not billed.

Starter is $39.99/mo with a 7-day trial. If you need to iterate quickly, start there and upgrade when traffic grows. See the Documentation for full parameter details.

Code you can drop into production

This example uses Python to classify a URL in real time, then route it through a block/allow policy and a simple cache. The code expects a JSON response with a categories array that includes category names and confidence scores.

import os
import time
import json
import requests
from typing import Dict, Any, List, Tuple, Optional

API_URL = "https://www.klazify.com/api/real_time_categorization"
API_KEY = os.getenv("KLAZIFY_API_KEY", "YOUR_API_KEY")

# Simple in-memory cache: domain -> (timestamp, result)
CACHE: Dict[str, Tuple[float, Dict[str, Any]]] = {}
CACHE_TTL_SECONDS = 6 * 60 * 60  # 6 hours; adjust to your freshness needs

# Example policy: map category prefixes to actions
# Keep your policies in config or DB so they can evolve without redeploys.
BLOCK_PREFIXES = [
    # Examples for illustration; use your own policy list
    "/Adult", "/Hate Content", "/Violence"
]
ALLOW_PREFIXES = [
    "/News", "/Business", "/Technology & Computing", "/Computers & Electronics"
]

def get_cached(domain: str) -> Optional[Dict[str, Any]]:
    entry = CACHE.get(domain)
    if not entry:
        return None
    ts, data = entry
    if (time.time() - ts) < CACHE_TTL_SECONDS:
        return data
    return None

def set_cache(domain: str, data: Dict[str, Any]) -> None:
    CACHE[domain] = (time.time(), data)

def classify_url(url: str) -> Dict[str, Any]:
    # Normalize a cache key to the registrable domain if you prefer domain-level caching
    cache_key = url
    cached = get_cached(cache_key)
    if cached:
        return cached

    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    payload = {"url": url}
    resp = requests.post(API_URL, headers=headers, data=json.dumps(payload), timeout=20)
    resp.raise_for_status()
    data = resp.json()
    set_cache(cache_key, data)
    return data

def pick_primary_category(categories: List[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
    if not categories:
        return None
    # Highest confidence first
    return sorted(categories, key=lambda c: c.get("confidence", 0), reverse=True)[0]

def action_for_categories(categories: List[Dict[str, Any]]) -> str:
    # Decide block/allow; implement overrides as needed
    # Use the top category to keep behavior predictable
    top = pick_primary_category(categories)
    if not top:
        return "review"

    name = top.get("name", "")
    for prefix in BLOCK_PREFIXES:
        if name.startswith(prefix):
            return "block"
    for prefix in ALLOW_PREFIXES:
        if name.startswith(prefix):
            return "allow"
    return "review"

if __name__ == "__main__":
    test_url = "https://cbsnews.com"
    result = classify_url(test_url)

    # The real-time response includes categories; some responses may also include helpful metadata
    domain = result.get("domain", {})
    categories = domain.get("categories", []) or result.get("categories", [])

    decision = action_for_categories(categories)
    print("Decision:", decision)
    print("Top category:", pick_primary_category(categories))

How to adapt this for your stack:

• Replace the policy lists with your organization’s allow/block taxonomy.
• Move the cache to Redis or your edge cache if you need distributed workers.
• Store the full response for observability and audits; keep a compact denormalized record for fast queries.

Understand the response fields you’ll actually use

The following is the official example response format. Use it to understand the structure of the domain object, categories, and ancillary information you may receive alongside categorization.

{
    "domain": {
        "categories": [
            {
                "confidence": 0.92,
                "name": "/Computers & Electronics/Consumer Electronics",
                "IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
            },
            {
                "confidence": 0.89,
                "name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
            }
        ],
        "social_media": null,
        "logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
    },
    "success": true,
    "objects": {
        "company": {
            "url": "https://www.apple.com/",
            "name": "Apple",
            "city": "Cupertino",
            "stateCode": "CA",
            "countryCode": "US",
            "employeesRange": "100K+",
            "revenue": 274515000000,
            "raised": null,
            "tags": [
                "E-commerce",
                "Consumer Electronics",
                "Mobile",
                "B2C"
            ],
            "tech": [
                "omniture_adobe_analytics",
                "atlassian_confluence",
                "successfactors",
                "apache_apex",
                "talend",
                "oracle_peoplesoft",
                "salesforce",
                "stripe",
                "dell_boomi_atomsphere",
                "gigya",
                "sage_50cloud",
                "quickbooks",
                "webmethods",
                "apache_tomcat",
                "alteryx",
                "tibco_rendezvous",
                "atlassian_jira",
                "..."
            ]
        }
    },
    "domain_registration_data": {
        "domain_age_date": "1987-02-19",
        "domain_age_days_ago": "13026",
        "domain_expiration_date": "2030-02-20",
        "domain_expiration_days_left": "123"
    },
    "similar_domains": [
        "bestbuy.com",
        "icloud.com",
        "microsoft.com",
        "macrumors.com",
        "google.com",
        "samsung.com",
        "twitter.com",
        "hp.com",
        "bhphotovideo.com",
        "dell.com"
    ]
}

What to use for real-time classification and policy decisions:

• domain.categories: Array of category objects. Each object includes:
  • name: Human-readable, hierarchical path (e.g., /Computers & Electronics/Consumer Electronics).
  • confidence: Float between 0 and 1. Use it to pick or rank categories.
  • IAB-...: Mapping to an IAB category code or path you can align with ad-tech and brand-safety controls.
• domain.logo_url: Useful for UI enrichment when you display classification results or build admin tooling.
• objects.company fields: Helpful for CRM enrichment and lead routing if you’re enriching a signup’s domain.

When calling the real-time endpoint, you will read categories with confidence for immediate decisions. Some responses may also include metadata such as a canonical domain and an updated timestamp depending on context; use those to record freshness in your logs. For a full description of the real-time response structure, consult the Documentation.

Turn categories into actions

Most production flows translate categories into a simple action. Here are patterns you can adapt without changing your data model.

• Blocklist enforcement (brand safety): Maintain a rules table keyed by category prefix (e.g., /Adult). On each classification, select the highest-confidence category and check it against your rules.
• Contextual targeting: Map IAB categories to your DSP or internal segments. A small table that maps from category path or code to your segment IDs is enough.
• Signup enrichment: Persist categories and company tags with the account record. Downstream scoring models can consume them alongside firmographics.
• Analytics & audit: Store the category and confidence plus a fetched_at timestamp. This lets you trend how categories drift over time and explain enforcement decisions.

Operational details that save time

Caching strategy

Cache by domain (or canonical domain) for a short TTL. For most use cases, 1–24 hours strikes a good freshness/cost balance. Use a separate TTL for high-churn sources if you suspect content changes more often. Record the time you fetched the category so downstream jobs can reason about freshness.

Batching and throughput

• Batch at your edge: Queue new domains and process with a worker pool that fans out concurrent POST requests.
• Deduplicate within a short window (e.g., 1 minute) to avoid refetching the same domain during traffic spikes.
• Backoff and retry on transient network errors. Failed or unreachable calls are not billed, but you still want resilience.

If you have a high-volume feed, keep a compact cache (domain → top category, confidence, fetched_at) and refresh on a rolling schedule in addition to your real-time path for cache misses.

Handling unknown or new domains

• If no categories are returned, route to “review” or a safe default (e.g., block for sensitive networks, allow for low-risk analytics).
• Log unknowns with a short retry cadence and a longer recheck schedule (e.g., 1h → 6h → 24h) because new sites may evolve quickly.
• Keep operators in the loop with a small UI that shows the last response, confidence, and any mapped policy outcome.

Mapping to your own taxonomy

Maintain a translation table that maps Klazify’s category names or IAB codes to your internal categories. Strategy:

• Normalize: Use the full hierarchical path (e.g., /Computers & Electronics/Consumer Electronics) as the lookup key.
• Fallbacks: If no exact match, step up the hierarchy (strip the rightmost node) until you resolve to a parent category.
• Confidence gating: Only create automatic mappings when confidence ≥ a threshold; send the rest to manual review to avoid noisy mappings.

Latency, timeouts, and observability

• Set a client timeout aligned with your SLA. If your ad decisioning must complete in 150 ms, front-load cache lookups and treat the API call as asynchronous enrichment or use a fallback policy when the timeout is hit.
• Emit metrics: success rate, median/95th percentile latency, and percentage of cached vs. live calls.
• Log response snippets: top category, confidence, and any relevant timestamps so you can explain decisions later.

Comparing implementation approaches

How to wire it into your pipeline

• CDN/edge: For ad or content delivery, check the cache first. If missing and you can’t block the request pipeline, enqueue a real-time call and proceed with a conservative default for this request; apply the decision on subsequent hits.
• Backend microservice: Wrap the POST call in a small service with circuit breakers, caching, and a metrics endpoint. Other services call yours rather than the third-party API directly.
• Data enrichment: A streaming job classifies new domains as they appear in logs or signups, writes categories and confidence to your warehouse, and backfills missing data during low-traffic windows.

Practical considerations before shipping

• Idempotency: De-duplicate calls per URL within a short window to reduce bursts.
• Storage schema: Keep fields for category name, confidence, fetched_at, and optionally the IAB mapping so you can interoperate with ad systems.
• Security: Store the API key in encrypted secrets. Scope IAM so only the calling service can read it.
• Governance: Version your policy rules. Log the policy version used for each decision to reproduce historical behavior.

Where to go next

Explore the broader capabilities—logo URLs, social detection, and company data—if you’re also enriching CRMs or building internal admin tools. See the MCP to understand how Klazify models domains, or head to the Documentation for endpoint details and field descriptions.

FAQ

• Is every real-time request actually live?
  Yes. The real-time endpoint analyzes the target on each request so you have up-to-date categories for immediate decisions.
• How should I handle rate and concurrency?
  Use a worker pool with bounded concurrency, retries with backoff for transient errors, and caching to cut redundant requests. Consult the documentation for current guidance and best practices.
• What if a site can’t be reached or returns no category?
  Failed or unreachable calls are not billed. In your pipeline, fall back to a default action (block, allow, or review) and schedule a retry at a later time.
• How do I map categories to my own taxonomy?
  Keep a translation table keyed by the category path or IAB mapping. If there’s no exact match, fall back to a parent category. Gate low-confidence results for human review.
• Can I use this for page-level decisions on large platforms?
  Yes—send the specific URL you need to classify. Cache wisely, and use a short TTL where content changes frequently.

Ready to try it with your own traffic? Create a free account and start classifying in minutes. Register and plug the real-time endpoint into your pipeline today.