Hacker News Scraper

What this Actor scrapes

This Actor fetches Hacker News story lists from any of the six available feeds — top, new, best, ask, show, or jobs — fans out to each individual story record, and writes one typed dataset row per story. The underlying Firebase API returns only item IDs at the feed level; we perform the full N+1 enrichment call per story and assemble the complete record before it hits your dataset.

You pick the feed and the result cap; we deliver clean, schema-validated rows on a schedule in JSON, CSV, or Excel — ready to pipe into Google Sheets, S3, a data warehouse, a webhook, or a RAG pipeline.

What we handle for you

• 🛡️ Browser fingerprint rotation — curl-cffi impersonates real Chrome, Firefox, and Safari TLS handshakes so we look like a browser, not a Python script.
• 🌐 Residential proxy rotation via Apify Proxy — fresh session and exit IP on every block or rate-limit response.
• 🔁 Retries with exponential backoff on 408 / 429 / 5xx — up to 5 attempts per request, Retry-After header honoured.
• 🧱 Rate-limit-aware pacing — when the target pushes back we slow down and surface a clear status message instead of silently returning an empty dataset.
• 🧊 Clean, typed dataset rows — Pydantic-validated, ISO-8601 timestamps, stable IDs, feed rank included.
• 💰 Pay-Per-Event pricing — you pay only for results that land in your dataset. No data, no charge.

Use cases

• Trend monitoring — diff top stories hourly to see which posts gain traction fastest.
• Comment-volume alerts — pipe rows into Slack when a story passes 100 comments.
• Lead gen for dev tools — surface Show HN launches that mention your stack and reach out early.
• Newsletter curation — feed the top 10 stories from the best feed into a weekly digest.
• ML training data — historical top-story metadata for score-prediction or topic-classification models.
• Show HN tracker — schedule a daily run against the show feed to watch new product launches.

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "feed": "top",
  "maxResults": 3,
  "includeText": false,
  "concurrency": 4,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

idtypetitleurlpermalinkbyscoredescendantstexttimeposted_atscraped_atrank

{
  "id": 39000000,
  "type": "story",
  "title": "Show HN: Devil Scrapes \u2014 public-data Apify Actors with honest pricing",
  "url": "https://apify.com/DevilScrapes",
  "permalink": "https://news.ycombinator.com/item?id=39000000",
  "by": "devilscrapes",
  "score": 142,
  "descendants": 33,
  "text": null,
  "time": 1747353600,
  "posted_at": "2026-05-15T20:00:00+00:00",
  "scraped_at": "2026-05-15T20:05:00+00:00",
  "rank": 1
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.