Certificate Transparency Subdomain Scraper

What this Actor scrapes

Every TLS certificate ever issued for a domain is logged forever in public Certificate Transparency logs — a firehose that crt.sh exposes as a free, keyless JSON endpoint. The catch: the raw payload is large, un-deduplicated, and carries no computed security signal. This Actor queries crt.sh per root domain and turns that firehose into a clean, capped dataset: one row per certificate (or per unique subdomain, your choice) with expiry countdown, expired/wildcard flags, and issuer metadata already computed — ready for an attack-surface inventory, a cert-hygiene sweep, or a daily subdomain-discovery job.

What we handle for you

• 🔎 One query per domain, whole log in one shot — crt.sh has no pagination; we fetch the complete result and process it in memory, so results land in one pass.
• 🧮 Computed security signals — days_until_expiry, is_expired, and is_wildcard come pre-computed, not left for you to derive from raw timestamps.
• 🧊 Deduplication, your way — one row per unique subdomain (keeping the newest cert), or one row per raw certificate/SAN entry if you want the closer-to-raw shape.
• 🚧 Egress guard built in — maxCertsPerDomain caps the response before a single row is built, so a domain with 10,000+ historical certificates can't blow your bill.
• 🔁 Retries with exponential backoff on 408 / 429 / 503 and network errors — a slow moment on crt.sh's shared database doesn't fail your run.
• 💰 Pay-Per-Event pricing — you only pay for certificates that land in your dataset. No data, no charge beyond the warm-up fee.

Use cases

• Attack-surface enumeration — pentest and bug bounty teams discovering subdomains from CT logs instead of brute-force DNS.
• Certificate hygiene monitoring — schedule daily runs across your own fleet and catch expiring or duplicate certificates.
• Brand protection — spot lookalike-domain certificate issuance for enforcement leads.
• Security research — track issuer patterns and wildcard usage across a domain's certificate history.

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "domains": [
    "github.com",
    "gitlab.com"
  ],
  "maxCertsPerDomain": 10,
  "deduplicateSubdomains": true,
  "includeExpired": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

domainsubdomaincertificate_idissuer_namecommon_nameserial_numbernot_beforenot_afterentry_timestampdays_until_expiryis_expiredis_wildcard

{
  "domain": "github.com",
  "subdomain": "api.github.com",
  "certificate_id": 12345678,
  "issuer_name": "C=US, O=DigiCert Inc, CN=DigiCert TLS RSA SHA256 2020 CA1",
  "common_name": "github.com",
  "serial_number": "0a1b2c3d4e5f",
  "not_before": "2026-01-15T00:00:00+00:00",
  "not_after": "2027-01-15T23:59:59+00:00",
  "entry_timestamp": "2026-01-15T08:12:03+00:00",
  "days_until_expiry": 132,
  "is_expired": false,
  "is_wildcard": false
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.

Limitations

• Certificate Transparency logs only see certificates issued by CAs that log to CT — self-signed or non-logged internal certificates never appear.
• No DNS resolution or liveness check — a discovered subdomain in the dataset may no longer resolve to a live host.
• No historical diffing between runs in v1 — compare successive dataset exports yourself to see what changed.