Open Library Scraper — Book Metadata in Bulk

What this Actor scrapes

Open Library is the Internet Archive's catalogue of 30M+ works — the open, canonical bibliographic source that most reliable book-metadata pipelines lean on. When Goodreads shut their developer API in 2020, they left a gap that five years later developers are still Googling around. Open Library fills it: no licensing hurdles, no API key friction, free bulk export — if you can navigate the pagination and handle the upstream's occasional rate-limiting.

This Actor turns a free-form query (title, author, ISBN, subject) into typed dataset rows with cover URL, subjects, edition count, and the canonical Open Library key. We pace requests against the upstream, retry on transient errors, and surface partial successes loudly — so your library, recommender, or research dataset gets the rows it expects.

What we handle for you

• 🛡️ Browser fingerprint rotation — curl-cffi impersonates real Chrome / Firefox / Safari TLS handshakes so the target sees a browser, not Python.
• 🌐 Residential proxy rotation via Apify Proxy — fresh session and exit IP on every block.
• 🔁 Retries with exponential backoff on 408 / 429 / 5xx — up to 5 attempts per page, Retry-After honoured.
• 🧱 Rate-limit-aware pacing — when the target pushes back, we slow down instead of getting banned.
• 🧊 Clean, typed dataset rows — Pydantic-validated, ISO-8601 timestamps, stable IDs, JSON / CSV / Excel export straight from the Apify Console.
• 💰 Pay-Per-Event pricing — you only pay for results that hit your dataset. No data, no charge.

Use cases

• Goodreads alternative API — rebuild the bibliographic data layer Goodreads took away in 2020. Title, authors, ISBNs, subjects, cover URL, edition count — the fields every book-rec app needs.
• ISBN lookup at bulk scale — enrich a CSV of book titles with ISBNs, authors, and covers in one run. Better unit economics than a per-request ISBN lookup API.
• Free book metadata API — feed a reading-list dashboard, a library catalogue app, or a fiction-RAG backend with structured Open Library data. No licensing restrictions on bibliographic metadata.
• Discovery pipelines — list every Asimov novel + edition count for a fan-site backend, or enumerate every title tagged "machine learning" for a curated reading list.
• Digital humanities — seed subject-tag corpora for distant-reading research, cultural-analytics, or AI-tutor curriculum ingestion.

Input

Paste this into the Apify Console, or send it as the run input over the API. Proxy settings are on by default; you rarely need to touch them.

{
  "searchQuery": "foundation asimov",
  "searchField": "all",
  "maxResults": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Output

One row per result, schema-validated before it is written. Export JSON, CSV, Excel or XML from the run, or read it over the API.

openlibrary_keytitlesubtitleauthorsfirst_publish_yearedition_countlanguagessubjectsisbnspublisherscover_idcover_url_lratings_averageratings_countebook_accesswork_url

{
  "openlibrary_key": "/works/OL471576W",
  "title": "Foundation",
  "authors": [
    "Isaac Asimov"
  ],
  "first_publish_year": 1951,
  "edition_count": 142,
  "work_url": "https://openlibrary.org/works/OL471576W"
}

Pricing

You pay only for results that land. Cap any run with maxTotalChargeUsd. See pricing & billing for worked examples.

Limitations

• Search uses Open Library's relevance ranking — for canonical bibliographic data (LCSH/Dewey), use a dedicated MARC source. Subjects are tags, not curated taxonomies.
• This Actor exports metadata only — titles, ISBNs, authors, subjects, cover URLs, publish years. It does not download book text or full-text content. For public-domain full-text, follow work_url to the Internet Archive reader.
• Open Library has thinner rating data than Goodreads. Treat ratings_average with caution for niche or older works.