HuggingFace Scraper — Models, Datasets & Spaces

What this Actor scrapes

Three repo types, one HuggingFace scraper:

1. Models — downloads, likes, pipeline tag, library name, tags, safetensors parameter count, GGUF file detection, and size-category bucketing. 2. Datasets — task categories, size categories, and language codes parsed from tag prefixes. 3. Spaces — SDK name and runtime stage (RUNNING / SLEEPING / STOPPED) in detail mode.

What we handle for you

• Three repo types in one Actor: model, dataset, space — pick via the repoType selector.
• Five sort fields: downloads, likes, trending, last_modified, created_at (all descending).
• Five filter modes: tag list, free-text search, author/org, single-repo deep fetch, or no filter (trending snapshot).
• Optional includeDetails mode — calls the per-repo detail endpoint to enrich with safetensors parameter counts, GGUF file detection, and Space runtime stage.
• GGUF detection flag and derived model_size_category bucket (<1B / 1B-7B / 7B-13B / 13B+) ready for downstream pricing or hardware-fit dashboards.
• Dataset tag prefixes auto-parsed into structured arrays: size_categories:, task_categories:, language:.
• Pydantic v2 input validation — at most one filter may be set; invalid input fails fast with a clear error before any network call.
• Exponential backoff on 429 and 503; honours Retry-After; max 5 attempts per endpoint call.
• Browser fingerprint impersonation via curl-cffi — no scraper-detectable headers leave the Actor.
• Companion to llm-pricing-monitor as part of the AI Stack Intelligence suite.

Use cases

• AI researcher trend tracking — pull the trending top-100 models weekly and feed a time-series dashboard tracking which model families dominate the Hub.
• Investor adoption monitoring — measure download velocity for specific model families (pipeline_tag=text-generation + library_name=transformers) to inform AI infrastructure investment theses.
• Fine-tuner catalog survey — enumerate every model under a pipeline_tag like image-segmentation to map the open-weights landscape before choosing a base model.
• Dataset discovery — filter datasets by task_categories and language to find labelled training corpora for a downstream NLP/vision/audio model.
• Hardware-fit analysis — use safetensors_total_params and model_size_category to filter models that fit a target memory budget before benchmarking.
• GGUF availability tracking — set includeDetails=true and filter on has_gguf=true to find quantized inference-ready models for llama.cpp or LM Studio.
• Space monitoring — capture which Spaces are RUNNING vs SLEEPING for a creator or topic, useful for community health dashboards.
• Content creator coverage — feed every model from a popular org like openai or meta-llama into a content pipeline for blog posts or YouTube videos.
• Track HuggingFace model downloads over time — schedule recurring runs to build a time-series of download counts and detect fast-movers before they hit mainstream coverage.

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "repoType": "model",
  "sort": "downloads",
  "maxResults": 100,
  "includeDetails": false,
  "useProxy": false
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

repo_typerepo_idrepo_ownerrepo_namerepo_urldownloadslikescreated_atlast_modifiedtagsgatedprivatepipeline_taglibrary_namesafetensors_total_paramsmodel_size_category

{
  "repo_type": "model",
  "repo_id": "openai/whisper-large-v3",
  "repo_owner": "openai",
  "repo_name": "whisper-large-v3",
  "repo_url": "https://huggingface.co/openai/whisper-large-v3",
  "downloads": 4932732,
  "likes": 5690,
  "created_at": "2023-11-07T18:41:14.000Z",
  "last_modified": "2024-08-12T10:20:10.000Z",
  "tags": [
    "transformers",
    "safetensors",
    "whisper",
    "automatic-speech-recognition"
  ],
  "gated": false,
  "private": false,
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "transformers",
  "safetensors_total_params": 1543490560,
  "model_size_category": "1B-7B",
  "has_gguf": false,
  "dataset_size_categories": null,
  "dataset_task_categories": null,
  "dataset_languages": null,
  "space_sdk": null,
  "space_runtime_stage": null,
  "scraped_at": "2026-05-16T12:00:00+00:00"
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.

Limitations

• Private and gated repos are not accessible. The unauthenticated public API only returns publicly visible data.
• Rate limit: 500 requests per 5 minutes (verified 2026-05-16 via ratelimit-policy header). At default page size 100, this allows ~10,000 list rows per 5-min window. Detail mode adds one request per row, halving throughput to ~250 enriched rows per minute.
• Spaces never have a downloads metric. The field is always null for repo_type=space — verified on both list and detail endpoints.
• Sparse Spaces list: repo_owner, last_modified, and space_runtime_stage require detail mode for Spaces.
• Safetensors and GGUF fields need detail mode. safetensors_total_params, model_size_category, and has_gguf are only populated when includeDetails=true.
• No cross-run deduplication. Re-running the same input returns the same repos with refreshed metadata. Use a downstream dedupe pass if you need uniqueness across runs.
• No model card or dataset card markdown content. Only structured metadata fields; the README body is excluded as too noisy for a structured dataset.
• No HuggingFace Inference API calls or model benchmarking. This Actor only scrapes catalog metadata, not model outputs.
• The Apify FREE tier retains run-scoped storage for 7 days only. For longer retention, export your dataset immediately or upgrade to a paid Apify plan.
• Use a trending snapshot weekly to track the rapidly-evolving model leaderboard. Set up an Apify Schedule for a recurring run.
• Cap maxResults to what you actually need. The HF Hub has 1M+ models; setting a sensible cap keeps cost and runtime predictable.
• Use detail mode sparingly. It is 2.5x the per-row cost and roughly 4x the per-row latency. Prefer list mode for catalog snapshots; flip to detail mode only when you need safetensors, GGUF, or runtime stage.
• Combine with llm-pricing-monitor to correlate open-weights releases on the Hub with hosted-API price moves.