Lemmy Scraper — Posts, Comments & Community Data

What this Actor scrapes

This Actor emits two row types into a single dataset, discriminated by the row_type column:

1. Post rows (row_type="post") — one row per post in the target community. Always emitted. 2. Comment rows (row_type="comment") — one row per comment on each post. Emitted only when includeComments is enabled.

What we handle for you

• 🛡️ We rotate browser fingerprints — curl-cffi impersonates real browser TLS (Chrome / Firefox / Safari) so the target sees real-browser handshakes, not Python.
• 🔁 We retry with exponential backoff on 408 / 429 / 503 responses and honour Retry-After. Up to 5 attempts per page before surfacing a clear error.
• 🌐 We rotate through Apify Proxy on connection failures — fresh session ID, fresh exit IP — so transient blocks don't abort your run.
• 🧱 We pace requests per instance to avoid triggering rate limits; partial successes surface a clear set_status_message — we never silently return an empty dataset.
• 🧊 We keep the dataset clean — Pydantic v2 validated rows, ISO 8601 timestamps, stable IDs, and a comment_parent_id field derived from Lemmy's path encoding so you can reconstruct the full thread tree without an extra API call.
• 💰 You pay only for results that land. No data → no charge beyond the small actor-start warm-up fee.
• Supports any Lemmy instance — lemmy.world, lemmy.ml, beehaw.org, sh.itjust.works, and any public instance running Lemmy v0.19.
• Two operating modes from one input — posts only, or posts + comments in the same dataset.
• Federated community syntax — pass memes@lemmy.world to scrape a remote community from any other instance, or asklemmy for a local community on the chosen instance.
• 17-token sort enum — Active, Hot, New, Old, Scaled, Controversial, MostComments, NewComments, TopHour, TopSixHour, TopTwelveHour, TopDay, TopWeek, TopMonth, TopYear, TopAll.
• Cursor-based post pagination + integer-page comment pagination — both verified against Lemmy v0.19 on live instances.
• Denormalised output — community metadata on every row, no joins needed for downstream analytics or CSV exports.
• Pydantic v2 input validation with named sort enum and range bounds; bare Top (invalid on v0.19) is rejected up-front before any network call.
• Pairs with bluesky-feed-posts and bluesky-starter-pack as the Federated Social Suite.

Use cases

• Reddit-alternative migration research — track how communities and engagement migrate from Reddit to Lemmy after policy changes; compare subscriber and post growth across instances.
• Newsroom monitoring — subscribe to journalism, politics, or breaking-news Lemmy communities and pipe the latest top posts to Slack or Google Sheets via Apify integrations.
• Brand monitoring on the fediverse — Lemmy is a growing channel for product complaints, support discussions, and competitor mentions outside the Reddit walled garden; this Actor surfaces them on a schedule.
• Academic federated-social research — Lemmy's public REST API makes it significantly more accessible for longitudinal community studies, sentiment tracking, or content-moderation research than platforms that gate their data.
• Community trend analysis — pull TopWeek posts across multiple communities and rank by score, comment count, or upvote ratio to benchmark community health.
• Comment-tree reconstruction — combine post rows with comment rows (joined by post_id) and the comment_parent_id field to rebuild the full discussion tree for NLP or moderation pipelines.
• NLP corpus building — Lemmy provides Reddit-shaped threaded conversation data useful for sentiment training, RAG pipelines, and discourse modelling without platform-specific OAuth hoops.

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "instanceUrl": "https://lemmy.world",
  "communityName": "asklemmy",
  "sort": "Hot",
  "maxPosts": 100,
  "includeComments": false,
  "maxCommentsPerPost": 50,
  "useProxy": false
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

row_typeinstance_urlcommunity_actor_idcommunity_namecommunity_titlecommunity_descriptioncommunity_subscriberscommunity_posts_countcommunity_localpost_idpost_ap_idpost_urlpost_titlepost_bodypost_external_urlpost_score

{
  "row_type": "post",
  "instance_url": "https://lemmy.world",
  "community_actor_id": "https://lemmy.world/c/asklemmy",
  "community_name": "asklemmy",
  "community_title": "Ask Lemmy",
  "community_description": "A Lemmy equivalent of Ask Reddit.",
  "community_subscribers": 39567,
  "community_posts_count": 8853,
  "community_local": true,
  "post_id": 46934299,
  "post_ap_id": "https://leminal.space/post/35446889",
  "post_url": "https://leminal.space/post/35446889",
  "post_title": "What's something that you feel genuinely sad about?",
  "post_body": null,
  "post_external_url": null,
  "post_score": 34,
  "post_upvotes": 38,
  "post_downvotes": 4,
  "post_comments_count": 18,
  "post_published": "2026-05-16T09:31:47.723504Z",
  "post_updated": null,
  "post_nsfw": false,
  "post_featured_community": false,
  "post_locked": false,
  "comment_id": null,
  "comment_ap_id": null,
  "comment_path": null,
  "comment_content": null,
  "comment_score": null,
  "comment_published": null,
  "comment_parent_id": null,
  "author_actor_id": "https://leminal.space/u/FosterMolasses",
  "author_name": "FosterMolasses",
  "author_display_name": null,
  "author_bot_account": false,
  "author_published": "2025-01-17T14:35:51.850105Z",
  "scraped_at": "2026-06-01T12:00:00.000Z"
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.

Limitations

• Public communities only. Private or access-restricted communities require authentication and are out of scope.
• One community per run. This Actor scrapes one community per run; multi-community or instance-wide listings need separate runs or a different Actor.
• No media download. Image, video, and external link URLs are captured in the row, but the Actor does not download the media content itself.
• No real-time streaming. The Actor takes a snapshot at run time; for live updates schedule recurring runs via Apify Schedules.
• 7-day default storage retention on the Apify FREE tier. Export your dataset immediately after the run or upgrade for longer retention.
• Lemmy v0.19 only. The Actor targets the v0.19 API surface and field paths. Pydantic extra="ignore" absorbs additive changes but breaking removes require a version update.
• Comment-tree depth is preserved via comment_path but the Actor does not return comments in a pre-nested structure — callers reconstruct the tree from comment_parent_id.
• Federation deduplication is the caller's responsibility. A post on c/news@lemmy.world may appear on multiple instances; use community_actor_id to deduplicate across multi-instance runs.