Synchronous Runs, Webhooks, and Scheduling on Apify

The async pattern in Run any DevilScrapes Actor via the API — start, poll, fetch — is the right default for anything that might run long. But for short, predictable jobs, or for hands-off recurring scrapes, Apify gives you two shortcuts: a synchronous run endpoint that returns data directly, and a scheduling/webhook layer so you don’t have to poll at all.

The synchronous endpoint

POST /v2/acts/{actorId}/run-sync-get-dataset-items starts a run and blocks the HTTP connection open until the run finishes, returning the dataset items directly in the response body — no run ID, no second request.

curl -X POST "https://api.apify.com/v2/acts/DevilScrapes~google-ads-transparency/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchDomains": ["nike.com"], "maxResults": 50}'

This returns the same clean JSON rows you’d get from GET /datasets/{id}/items?format=json&clean=1, in one call.

When to use it, and when not to

Use run-sync-get-dataset-items when:

• The job reliably finishes in well under a minute (a small maxResults, a single-page lookup).
• You’re calling it from something with a short request timeout budget of its own — a serverless function, a chat-bot backend, an n8n HTTP node — and want the simplest possible integration.

Avoid it when:

• The Actor might run for minutes on a large input. Most HTTP clients, proxies, and load balancers time out long-held connections well before a big scrape finishes, and you’ll get a client-side timeout even though the run itself would have succeeded.
• You need to react to partial progress, retries, or a FAILED status with a specific error message — the sync endpoint gives you data or an error, not the intermediate run lifecycle.

For anything uncertain in duration, use the async POST /runs pattern and either poll or set up a webhook (below).

Query params worth setting

• timeout — seconds to hold the connection before giving up (still bills the run; it just stops waiting).
• memory — override the Actor’s default memory allocation for this call.
• maxItems / maxTotalChargeUsd — cap how much the run returns or costs before it stops.

Webhooks: get notified instead of polling

Register a webhook once, and Apify posts a JSON payload to your URL whenever a matching event fires, on any run of the Actor (or a specific run, if you scope it).

curl -X POST "https://api.apify.com/v2/webhooks" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED"],
    "condition": { "actorId": "DevilScrapes~google-ads-transparency" },
    "requestUrl": "https://your-app.example.com/webhooks/apify"
  }'

Common eventTypes: ACTOR.RUN.SUCCEEDED, ACTOR.RUN.FAILED, ACTOR.RUN.ABORTED, ACTOR.RUN.TIMED_OUT. The payload includes the run ID and dataset ID, so your handler’s job is usually just “fetch the dataset items now that this run is done” — the same GET /datasets/{id}/items call from the previous page.

Verify the signature. Apify signs webhook payloads with the Apify-Webhook-Request-Signature header. Check it against your webhook’s secret before trusting the payload — don’t act on an unverified POST to a public endpoint.

Scheduling recurring runs

If you need an Actor to run on a cadence — daily price checks, hourly feed pulls — use Apify Schedules instead of your own cron hitting the API:

curl -X POST "https://api.apify.com/v2/schedules" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "nike-ads-daily",
    "cronExpression": "0 6 * * *",
    "actions": [{
      "type": "RUN_ACTOR",
      "actorId": "DevilScrapes~google-ads-transparency",
      "runInput": { "body": "{\"searchDomains\": [\"nike.com\"], \"maxResults\": 200}", "contentType": "application/json" }
    }]
  }'

This runs the Actor at 06:00 UTC every day with a fixed input, billed the same Pay-Per-Event way as any manual run. Combine a schedule with a webhook to get a fully hands-off pipeline: schedule triggers the run, webhook fires when it’s done, your handler pulls the dataset and does whatever’s next (load a warehouse, send a Slack message, refresh a dashboard).

You can manage schedules from the Apify Console too, under an Actor’s Schedules tab, if you’d rather not script it.

Polling tips, if you’re not using a webhook

If you’re sticking with manual polling from the async pattern:

• Back off — check every 2–3 seconds for short jobs, every 10–30 seconds for longer ones. Hammering the run-status endpoint every 100ms wastes your own rate-limit budget for nothing.
• Stop polling once status is any terminal value, not just SUCCEEDED — FAILED, ABORTED, and TIMED_OUT are all final and won’t change.
• Read statusMessage on FAILED or partial runs — our Actors set a specific status message (e.g. “Scraped 142/200 results before the target rate-limited”) rather than leaving you to guess why a run stopped short.

FAQ

Will run-sync-get-dataset-items time out on a large scrape?

It can — both your client and Apify’s own gateway have connection-hold limits. For anything beyond a small, fast lookup, use the async POST /runs endpoint and either poll or register a webhook instead.

Do webhooks cost extra?

No, registering and receiving webhooks doesn’t add to your Pay-Per-Event bill. You’re billed for the Actor run itself, exactly as if you’d triggered it any other way.

Can I schedule a run with different inputs each time?

Not from a single static schedule — a schedule holds one fixed runInput. For varying inputs on a cadence, use an external scheduler (cron, a CI job, n8n’s schedule trigger) to call the async run endpoint with a fresh input each time.

How do I verify a webhook payload is really from Apify?

Check the Apify-Webhook-Request-Signature header against the signing secret you set when creating the webhook. Reject anything that doesn’t match before processing the payload.

What happens if my webhook endpoint is down when the event fires?

Apify retries webhook delivery on failure for a period of time before giving up. If you need a guarantee, don’t rely solely on the webhook — poll the run status as a fallback for anything business-critical.