Stack Overflow Scraper — Stack Exchange Questions

What this Actor scrapes

The Stack Exchange network (api.stackexchange.com/2.3) covers every site — Stack Overflow, Server Fault, Super User, Cross Validated, plus 170+ topic communities. This Actor wraps the questions endpoint, paginates safely through the backoff field, rotates API quota keys, and writes one clean row per question with body, tags, and key metadata. No quota overruns, no partial failures swept under the rug.

Stack Overflow is the world's largest developer Q&A corpus. Post the SO data dump pause, this is the freshest pipeline you can run without getting tangled in the CC BY-SA attribution maze yourself — we include posted_at + a stable question_id so your downstream attribution is always correct.

What we handle for you

• 🛡️ Browser fingerprint rotation — curl-cffi impersonates real Chrome / Firefox / Safari TLS handshakes so the target sees a browser, not a Python script.
• 🌐 Residential proxy rotation via Apify Proxy — fresh session and exit IP on every block, so your quota never drains from a single flagged address.
• 🔁 Retries with exponential backoff on 408 / 429 / 5xx — up to 5 attempts per page, Retry-After honoured, no silent empty results.
• 🧱 Rate-limit-aware pacing — when Stack Exchange pushes back, we slow down instead of burning your daily quota.
• 🧊 Clean, typed dataset rows — Pydantic-validated, ISO-8601 timestamps, stable IDs, JSON / CSV / Excel export straight from the Apify Console.
• 💰 Pay-Per-Event pricing — you only pay for results that hit your dataset. No data, no charge (beyond the tiny warm-up fee).

Use cases

• RAG corpus pipeline — feed Q&A bodies from your product's tag into a LangChain / LlamaIndex vector store for a domain copilot.
• DevRel feedback signal — daily diffs on unanswered questions in your tag to surface gaps before users churn.
• Competitor-tag intelligence — compare question volume and score trends across react vs vue vs angular over time.
• Help-center seed — pull the top-200 voted questions per tag to pre-populate an internal knowledge base.
• Recruiter outreach — extract active askers from a niche tag, score by reputation (via the user endpoint).
• AI training dataset — build a deduped, fresh alternative to the paused SO data dump; each row includes attribution fields required by CC BY-SA 4.0.

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "site": "stackoverflow",
  "mode": "tagged",
  "tags": [
    "python"
  ],
  "sortBy": "votes",
  "maxResults": 100,
  "includeBody": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

question_idsitetitlebody_htmltagsscoreview_countanswer_countis_answeredaccepted_answer_idlinkowner_user_idowner_display_namecreation_datelast_activity_dateposted_at

{
  "question_id": 1234567,
  "site": "stackoverflow",
  "title": "How do I close a connection cleanly in asyncio?",
  "body_html": "<p>I'm trying to gracefully shut down an asyncio server\u2026</p>",
  "tags": [
    "python",
    "asyncio"
  ],
  "score": 142,
  "view_count": 48300,
  "answer_count": 3,
  "is_answered": true,
  "accepted_answer_id": 1234570,
  "link": "https://stackoverflow.com/questions/1234567/...",
  "owner_user_id": 987654,
  "owner_display_name": "asyncio_dev",
  "creation_date": 1609459200,
  "last_activity_date": 1712345678,
  "posted_at": "2021-01-01T00:00:00Z",
  "scraped_at": "2026-06-01T10:00:00Z"
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.

Limitations

• Question bodies only — comments, voting graphs, revision history, and answer bodies are not in scope for this Actor.
• Search ranking is Stack Exchange's own, which differs from the site's visible UI sort in subtle ways.
• Daily quota — without an API key you get 300 requests/day (each page = 1 request, 100 questions/page). With a free key from stackapps.com you get 10 000 requests/day. For very large jobs, plan your daily budget accordingly.
• Deleted users — some owner_user_id and owner_display_name fields will be null where the asker's account was removed.