How to Screen a Website by Category

November 03, 2026
How to Screen a Website by Category

You need to automatically decide whether to block, allow, or enrich a website based on its content category. By the end of this guide, you’ll be able to call a single API, read an IAB category and confidence, and plug that result into a filter, signup enrichment step, or ad audit workflow.

Why Klazify is the best fit for screening by category

For screening websites by topic (for example, determining if something should be treated as news, adult content, or finance), the details that matter are accuracy, coverage, and the ease of putting the result into production. Klazify focuses on analyzing real page content, mapping results to IAB taxonomy, and returning everything you need—from categories to logo and company data—in one call. This keeps your category screen fast and actionable.

Illustration: How to Screen a Website by Category
  • Accurate website categorization using AI: Klazify evaluates on-page content and context, not just metadata. This helps correctly identify nuanced sites where the title or domain name isn’t enough to decide how to treat the content.
  • Global coverage: Multilingual analysis means you can apply the same category policy worldwide without hand-coded rules by locale.
  • Real-time classification: Instead of relying on stale offline lists, you get up-to-date categorization suitable for dynamic content and recently launched domains.
  • Industry-level categories: IAB taxonomy support provides a standard, granular way to plug category flags directly into brand safety rules, content filters, and ad placement logic.
  • Simple API integration: A single REST endpoint delivers category, logo, social profiles, company data, technologies detected, domain registration data, and similar domains—useful for enrichment, deduping, and cross-checks.
  • Compliance and filtering: You can programmatically allowlist/denylist based on IAB-aligned categories and confidence, enabling brand safety, parental control, or workplace policy enforcement.

If you need a quick start, create a free account and get your API key at klazify.com or go straight to Register. The Starter plan is $39.99/mo with a 7-day trial. Failed or unreachable calls are not billed.

End-to-end flow: from a URL to a category decision

Below is the minimal flow you can drop into any backend or data pipeline:

  1. Send the URL to Klazify’s categorize endpoint.
  2. Read the top category name and confidence score.
  3. Compare the category against your allowlist/denylist (IAB-aligned) and apply your confidence threshold.
  4. Cache the result per domain so subsequent traffic is instant.
  5. Optionally save additional fields (logo, social profiles, company metadata, tech) for analytics or CRM enrichment.

API request: get the category for a website

Send a POST request with a JSON body that contains the URL to classify. Use your API key in the Authorization header.

curl -X POST "https://www.klazify.com/api/categorize" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://cbsnews.com"}'

Keep your requests idempotent by always normalizing the URL (e.g., lowercase hostname, strip tracking params if your policy requires it). Store results by domain to maximize cache hits.

Official sample response and the fields you’ll use

This is the official example response format. It shows how category data, logo, company information, domain registration data, and similar domains appear in a single payload.


{
    "domain": {
        "categories": [
            {
                "confidence": 0.92,
                "name": "/Computers & Electronics/Consumer Electronics",
                "IAB-632-596": "Consumer Electronics/Technology & Computing/Consumer Electronics"
            },
            {
                "confidence": 0.89,
                "name": "/Internet & Telecom/Mobile & Wireless/Mobile Phones"
            }
        ],
        "social_media": null,
        "logo_url": "https://klazify.s3.amazonaws.com/2110787991611585019600ed5fb1d1300.04730104.png"
    },
    "success": true,
    "objects": {
        "company": {
            "url": "https://www.apple.com/",
            "name": "Apple",
            "city": "Cupertino",
            "stateCode": "CA",
            "countryCode": "US",
            "employeesRange": "100K+",
            "revenue": 274515000000,
            "raised": null,
            "tags": [
                "E-commerce",
                "Consumer Electronics",
                "Mobile",
                "B2C"
            ],
            "tech": [
                "omniture_adobe_analytics",
                "atlassian_confluence",
                "successfactors",
                "apache_apex",
                "talend",
                "oracle_peoplesoft",
                "salesforce",
                "stripe",
                "dell_boomi_atomsphere",
                "gigya",
                "sage_50cloud",
                "quickbooks",
                "webmethods",
                "apache_tomcat",
                "alteryx",
                "tibco_rendezvous",
                "atlassian_jira",
                "..."
            ]
        }
    },
    "domain_registration_data": {
        "domain_age_date": "1987-02-19",
        "domain_age_days_ago": "13026",
        "domain_expiration_date": "2030-02-20",
        "domain_expiration_days_left": "123"
    },
    "similar_domains": [
        "bestbuy.com",
        "icloud.com",
        "microsoft.com",
        "macrumors.com",
        "google.com",
        "samsung.com",
        "twitter.com",
        "hp.com",
        "bhphotovideo.com",
        "dell.com"
    ]
}

How to use it:

  • domain.categories: Read the highest-confidence category. The field name contains the hierarchical path (e.g., /Category/Subcategory). If present, use the IAB mapping for standardized policy enforcement.
  • domain.categories[].confidence: Float in [0, 1]. Set your acceptance threshold (e.g., 0.80) before enforcing allow/deny rules.
  • domain.logo_url: Useful for UI in admin consoles, review queues, or CRM enrichment.
  • objects.company: Optional enrichment—store name, location fields, tags, and tech for operations like deduping or segmentation. Do not rely on these for category policy; they’re enrichment.
  • domain_registration_data: Dates help you flag very new domains for extra review in security or ad vetting use cases.
  • similar_domains: Helpful for proactive audits (e.g., if a domain is denied, queue similar ones for scheduled screening).

Code example: classify and enforce a category policy

This Python example demonstrates calling the endpoint, reading the top category and confidence, and making a decision. Replace YOUR_API_KEY with your actual token.

import json
import os
import requests

API_URL = "https://www.klazify.com/api/categorize"
API_KEY = os.getenv("KLAZIFY_API_KEY", "YOUR_API_KEY")

def classify_url(url: str):
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    }
    payload = {"url": url}
    resp = requests.post(API_URL, headers=headers, data=json.dumps(payload), timeout=20)
    resp.raise_for_status()
    data = resp.json()
    return data

def top_category_and_confidence(data: dict):
    cats = data.get("domain", {}).get("categories", []) or []
    if not cats:
        return None, 0.0
    # categories are already confidence-scored; assume first is the highest or sort if needed
    cats_sorted = sorted(cats, key=lambda c: c.get("confidence", 0), reverse=True)
    top = cats_sorted[0]
    return top.get("name"), float(top.get("confidence", 0))

def decision_from_category(cat_name: str, confidence: float, denylist: set, min_conf: float = 0.8):
    if not cat_name:
        return "review"
    if confidence < min_conf:
        return "review"
    # Example: block if in denylist; otherwise allow
    # You can compare by prefix to match hierarchical paths
    if any(cat_name.startswith(d) for d in denylist):
        return "block"
    return "allow"

if __name__ == "__main__":
    url_to_check = "https://example.com"
    data = classify_url(url_to_check)
    cat_name, conf = top_category_and_confidence(data)
    denylist = {
        # e.g., "/News" or "/Adult" or "/Gambling", depending on your policy taxonomy
    }
    action = decision_from_category(cat_name, conf, denylist, min_conf=0.8)
    print("url:", url_to_check)
    print("category:", cat_name)
    print("confidence:", conf)
    print("action:", action)

    # Optional enrichment:
    logo = data.get("domain", {}).get("logo_url")
    company = data.get("objects", {}).get("company", {})
    print("logo_url:", logo)
    print("company_name:", company.get("name"))

Adapt the denylist/allowlist to your compliance or brand safety framework. If your policy uses your own labels instead of IAB terms, map the returned category to your internal labels before decisioning.

Designing robust screening: key decisions

1) Thresholds and fallback paths

  • Confidence threshold: Pick a minimum acceptable confidence (for production, 0.75–0.90 is a typical range—tune with real data). Below that threshold, send to “manual review” or temporary “quarantine” logic.
  • No-category fallback: If domain.categories is empty or missing, retry after a brief delay, or log and continue as “unknown” pending review. You can also use domain_registration_data to weight risk on new domains.

2) Hierarchical matching

  • When you compare categories to your policy, use prefix matching on the hierarchical path. For example, if you block a parent like /Internet & Telecom/Mobile & Wireless/, all its children match automatically.
  • If you require granular exceptions, add explicit allow rules for specific subcategories that override broader denies.

3) Caching strategy

  • Cache by eTLD+1 (e.g., example.co.uk) or by full hostname if subdomains vary widely.
  • TTL policy: Short TTLs for fast-moving domains; longer TTLs for stable corporate sites. Consider refreshing asynchronously.
  • Invalidate cache when your policy changes or when high-traffic domains are flagged for recheck during audits.

4) Handling new or unknown domains

  • Retry logic: On transient HTTP failures or timeouts, retry with exponential backoff. Failed or unreachable calls are not billed.
  • Queueing: For bulk ingestion, queue unknown domains for asynchronous classification before they appear in production logs.
  • Manual pipeline: Provide a review UI fed by the API’s logo_url, company name, and similar_domains to accelerate manual decisions.

5) Batching and throughput

  • Parallelization: Send concurrent POST requests within your service’s limits to achieve throughput. If you are enriching backlog data, shard by domain hash to spread load.
  • De-duplication: Deduplicate domains within a job batch to improve performance and reduce calls.
  • Backpressure: If your system hits internal rate thresholds, enqueue and drain in waves; use caching to avoid re-hit.

Using category results in real scenarios

Block or allow traffic by category

In a corporate network filter, you can deny traffic when the top category matches your denylist and the confidence exceeds your minimum. Log the category and confidence for audit trails. For unknown or low-confidence results, route to a manual review queue.

Enrich signups or leads

When a user signs up with a company email or a domain, call the API and store:

  • Top category and confidence: segment the account for onboarding or sales routing.
  • objects.company fields: prefill company name, location, size range, and tags in CRM.
  • logo_url: use in dashboards for quick account recognition.

Audit where ads run

For ad operations, attach the IAB-mapped category to each impression domain. Then create rules that allow or deny certain groups. If an impression occurred on a site outside your approved taxonomies, trigger a review and check similar_domains to find related inventory to audit next.

Mapping to your internal taxonomy

If your organization already uses a custom set of labels, map the IAB taxonomy (and the category path in name) to those labels. Maintain this mapping in a table and use prefix-based rules to keep maintenance small.

Returned field Example content How to use it Notes
domain.categories[].name /Internet & Telecom/Mobile & Wireless/Mobile Phones Match with your taxonomy by prefix or mapping table Use hierarchical paths to minimize rule count
domain.categories[].confidence 0.89 Apply thresholds and review routing Store for audit and continuous tuning
IAB mapping (when present) Consumer Electronics/Technology & Computing/Consumer Electronics Directly align with IAB-based policies Prefer IAB mapping for ad safety rules
domain.logo_url https://...png Show in dashboards and review tools Cache the asset or proxy via your CDN
objects.company.* name, location, tags, tech CRM enrichment, dedupe, routing Not required for policy enforcement
domain_registration_data.* age, expiration dates Risk flags for very new domains Combine with category for security use cases
similar_domains [...] Proactive audits and allow/deny propagation Useful for inventory curation

Data pipeline and storage patterns

  • Normalize input: Lowercase domain, strip trailing slashes, optionally canonicalize URLs before calling.
  • Store domain, category path, confidence, IAB mapping (if present), timestamp, and a hash of the content you classified if needed for change detection.
  • Cache per domain with TTL; refresh asynchronously for your top N traffic domains.
  • Create a denormalized reporting table keyed by domain and day for analytics and ad safety audits.
  • Surface low-confidence classifications to an internal review queue with logo_url and company.name for faster triage.

Operational notes developers ask about

  • Authentication: Use Authorization: Bearer YOUR_API_KEY. Rotate keys via your secrets manager and CI/CD.
  • Rate and batching: The categorize request is per URL. For bulk jobs, parallelize safely and introduce backpressure. Cache aggressively to reduce reclassification.
  • Latency: Run the request server-side close to your application. Install circuit breakers and retries with exponential backoff.
  • Error handling: Treat 4xx/5xx and network timeouts as retryable (up to a point). Failed or unreachable calls are not billed.
  • Change management: Keep a versioned policy config (denylist/allowlist, thresholds, mapping table). Re-run audits when policies change.

Putting it all together: policy-as-code

Implement category rules as code in your gateway or middleware. For example:

  • A YAML or JSON file defines denylist prefixes and required minimum confidence.
  • Your service resolves the domain from the request, fetches (or caches) the category, evaluates against config, and returns allow/block.
  • Results are logged with category, confidence, and the policy version used for the decision.

This approach lets you review and roll back policy changes, A/B test thresholds, and generate weekly audit reports on block/allow distributions. The same configuration can also drive enrichment and ad inventory curation jobs.

Quality checks before production

  • Sample tests: Run a curated dataset of allowed and denied domains through your pipeline and verify decisions and confidence histograms.
  • Edge cases: Subdomains with very different content from the root; redirected URLs; internationalized domains; new sites with sparse content.
  • Observability: Add metrics for request volume, error rates, median/95th percentile latency, cache hit rate, and distribution of confidence scores.
  • Review cadence: Schedule reclassification of your top domains weekly or monthly to catch taxonomy drift and content changes.

Where to go next

Use the API to screen, enrich, and audit in one step. Explore the full set of fields and behaviors in the Documentation, and review your integration posture in the MCP. You can get started in minutes at klazify.com and create a free account with a 7-day trial.

FAQ

How should I pick a confidence threshold?
Start by inspecting confidence distributions on your real traffic. Choose a threshold where false positives are minimized for your critical categories. Revisit quarterly and adjust based on drift.

Can I classify pages (URLs) instead of entire domains?
Yes—send the full URL in the same request payload. This is useful for sites with mixed content across different sections.

What happens if the API can’t reach the site?
The call won’t be billed. Implement retries and a fallback path (e.g., manual review, temporary allow/deny depending on your risk model).

How do I keep performance high at scale?
Cache per domain, deduplicate requests, parallelize safely, and refresh asynchronously for your top domains. Log and measure cache hit rates.

Can I use the company and tech fields for routing leads?
Yes. Store objects.company fields and tech signals alongside the category for routing, segmentation, or scoring. Keep policy enforcement based on category and confidence to avoid mixing heuristics.

Ready to implement category screening in your stack? Create your API key now and start classifying with a 7-day trial: Register.

Ready to use Klazify?

Start classifying websites, enriching company data, and exploring web intelligence.

Get Started Free