CKAN Open Data Portal Scraper

What this Actor scrapes

deployed open-data portal software in the world, and every deployment exposes the same standardized Action API — package_search, package_show, organization_list — regardless of which government or city runs it. data.gov.uk, open.canada.ca, data.gov, the EU portal, and most national and city open-data catalogues all speak this protocol.

This Actor is a generic, cross-portal harvester, not a single-country scraper. You supply a list of {portal_url, action} requests, and we return every matching record from every request in one run — dataset metadata, resource listings, or organization rosters, whatever shape that portal's catalogue happens to publish.

What we handle for you

• 🔁 We retry with exponential backoff on 429 and 5xx responses —
• 📄 We page every search for you — rows/start walked automatically
• 🧩 We fan out across portals in one run — mix a UK search and a
• 🧱 We isolate failures per request — CKAN's native success: false
• 🧊 We keep the dataset clean — Pydantic-validated rows, ISO-8601
• 💰 You pay only for results that land. No data → no charge beyond the

Use cases

• Civic analytics — pull a national or city open-data catalogue on a
• Journalism / research — cross-reference multiple government portals
• Data engineering — feed dataset metadata into a pipeline or catalogue
• Academic / policy research — bulk-export any CKAN portal's catalogue
• Compliance / transparency monitoring — watch an organization's

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "requests": [
    {
      "portal_url": "https://data.gov.uk",
      "action": "package_search",
      "rows": 5,
      "label": "data.gov.uk sample search"
    },
    {
      "portal_url": "https://open.canada.ca/data",
      "action": "package_search",
      "rows": 5,
      "label": "open.canada.ca sample search"
    }
  ],
  "proxy_configuration": {
    "useApifyProxy": true
  }
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

portal_urlactionquery_usedrequest_labelrecord_idscraped_atdata

{
  "portal_url": "https://data.gov.uk",
  "action": "package_search",
  "query_used": null,
  "request_label": "data.gov.uk sample search",
  "record_id": "land-registry-price-paid-data",
  "scraped_at": "2026-09-01T09:12:44.000Z",
  "data": {
    "id": "5a67e4a1-0000-0000-0000-000000000000",
    "name": "land-registry-price-paid-data",
    "title": "Price Paid Data",
    "organization": {
      "name": "land-registry",
      "title": "HM Land Registry"
    },
    "num_resources": 4
  }
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.