itch-library-pipeline

Get the books you own on itch.io — TTRPGs, ebooks, comics — into your self-hosted reading stack (BookLore + Komga) with real cover art and metadata, instead of downloading and sorting hundreds of files by hand.

It does the whole path — download → sort → add covers & blurbs → keep new arrivals fresh — and is built to be safe to try: every step that moves files or writes to your library is a preview by default and only changes anything once you add --commit. It dedupes against what you already own so you don't end up with doubles, and it never downloads games, soundtracks, or software — only books.

New to all this? No problem. The Quick start below is copy-paste steps, and every step has a click-to-expand explainer telling you what the command does and why.

---

Quick start

The pipeline has seven modes, but you'll usually run a few in order: fetch your books, sort them into libraries, then enrich them (report → apply) with covers and descriptions. The safe habit throughout: run the preview, read the CSV it writes, then re-run with --commit.

First time using a terminal? Start here

A "terminal" is a window where you type commands instead of clicking buttons.

• Windows: install Docker Desktop, then open PowerShell (search for it in the Start menu).
• macOS: install Docker Desktop, then open Terminal (in Applications → Utilities).
• Linux: you likely already have a terminal and may already have Docker.

To run a command, copy it, paste it into the terminal window, and press Enter. "Docker" is just a tool that runs this program in a tidy box, so you don't have to install Python or anything else yourself. This tool talks to your existing BookLore and Komga servers, so those need to be running already.

1. Get the code and build the image

git clone https://github.com/you/itch-library-pipeline.git
cd itch-library-pipeline
cp .env.example .env
docker compose build

What did that do?

• git clone downloads a copy of this project; cd … ("change directory") moves you into the folder it made, so the rest of the commands run in the right place. Replace the URL with wherever you actually got this (e.g. your own Forgejo/GitHub address).
• cp .env.example .env makes your personal settings file from the template. You'll fill it in next — see Personalize it to your setup.
• docker compose build packages the tool into a runnable image once, so every later command starts instantly. (Docker can't build and run in one go for this kind of tool, so this is a one-time step you repeat only when the code changes.)

2. Fill in .env

Open .env in a text editor and fill in the values for the modes you'll use. At minimum, for the quick start you want your itch API key, your library folder paths, and your BookLore database password. See Personalize it to your setup for exactly what to set and how to find each value.

3. Download the books you own (fetch)

docker compose run --rm pipeline fetch

What does this do?

fetch logs into itch.io with your API key and downloads every book in your library — TTRPGs, ebooks, comics — into your staging folder, skipping video games, soundtracks, and software. It only ever writes to the staging folder, so it's safe to run.

docker compose run --rm pipeline means "run the pipeline service once and throw the container away afterwards (--rm)"; the word after it (fetch) is the mode.

Note on bundles: itch-dl (the downloader underneath) only sees games that are actually in your library — itch bundle games don't count as owned until you claim them. If you bought a bundle, claim its games first (a Tampermonkey/Violentmonkey "bundle claimer" userscript, e.g. greasyfork 446312, run on the bundle's download page, does this in bulk), then run fetch.

4. Sort them into your libraries — preview first

docker compose run --rm pipeline sort

This writes out/sort_plan.csv and moves nothing. Open that file, check the planned moves (flip any move to skip if you disagree), then commit:

docker compose run --rm pipeline sort --commit

What's it sorting, and why preview first?

sort looks at each downloaded book, reads its itch.io category, and decides where it belongs: a Physical game → your TTRPG library, a Book → your ebook library, a Comic → your Komga comics. It also dedupes against what's already in those libraries so you don't get a second copy of something you own.

Without --commit it's a dry run: it just writes the plan to out/sort_plan.csv. Reviewing that file before committing is the safe habit — the decision column on each row is move or skip, and you can change any of them before the real run.

5. Add covers and descriptions (report → apply)

First build the review file (read-only, touches nothing):

docker compose run --rm pipeline report

This writes out/report.csv. Open it and look at the decision column — confident matches are pre-set to apply, uncertain ones to review, weak ones to skip. Change any you like, then write the approved rows into BookLore:

docker compose run --rm pipeline apply --commit

What's happening here, and what gets written?

report matches every book in your TTRPG library against itch.io and DriveThruRPG and scores each match on both title and author. It sorts results into confidence tiers — HIGH (pre-marked apply), MEDIUM (review), LOW (skip) — and writes them to out/report.csv. It writes only the CSV; nothing in BookLore changes.

apply --commit reads that CSV back and writes the cover art, title, description, and rating into BookLore — but only for rows whose decision says apply. It also locks every field it sets, so a later BookLore metadata refresh won't overwrite your good covers. Run apply without --commit first if you want to see what it would write before it writes.

6. (Optional) Keep new books enriched automatically (watch)

docker compose run -d --name itch-watch pipeline watch

What does watch mode do?

watch is the one mode that runs forever (the -d runs it in the background). It notices when BookLore imports a new book and enriches it on the spot — no scheduled full-library scan. It only auto-writes matches at or above your confidence threshold (WATCH_MIN_CONFIDENCE); anything less sure is just logged for you to handle manually.

On first start it records all your current books as "already seen" so it never blindly itch-matches your existing backlog — only books added after it starts. (Run the existing backlog through report/apply instead.) Watch its output with docker compose logs -f itch-watch.

The other two modes: comics and series

• comics — pushes itch.io metadata (covers, summaries) into Komga for the comics that sort filed there. Preview with docker compose run --rm pipeline comics, then add --commit.
• series — groups books that belong together (a game's rulebook + sheets, or a base game plus its expansions/playbooks) under one BookLore series, so they stack into a single shelf entry. Writes out/series_plan.csv as a preview; add --commit to write the tags.

---

Personalize it to your setup

Everything below lives in your .env file. Fill in only the lines for the modes you'll actually use (the comments at the top of .env.example say which vars each mode needs).

Your itch.io API key (ITCH_API_KEY) — needed for fetch.

How do I find my itch.io API key?

Log in to itch.io, go to Settings → API keys (or directly to https://itch.io/user/settings/api-keys), and Generate new API key. Copy it into ITCH_API_KEY= in your .env.

Your library folder paths — where the tool puts sorted books, and where it reads your existing ones to dedupe and enrich.

How do I find my library paths?

These are the folders on your server/computer that hold each library. Set them in .env:

• STAGING_DIR — an empty working folder for freshly downloaded files (e.g. /srv/.../itch-staging).
• SORT_TTRPG_DIR, SORT_EBOOK_DIR, SORT_COMICS_DIR — your real TTRPG, ebook, and comics library folders. Use the path as the machine running Docker sees it (on a NAS/server, the host path; on a desktop, e.g. D:/Media/TTRPG).
• BOOKLORE_DATA_DIR — the host folder that BookLore itself mounts at /app/data (that's where apply writes cover image files). Check your BookLore container's volume settings.

Tip: open the folder in your file manager and copy the path from the address bar.

Your BookLore database details — report/apply/watch/series read BookLore's database directly.

How do I find the BookLore DB password, library id, and network?

• BOOKLORE_DB_PASSWORD — the MariaDB password from your BookLore stack's own config/.env (the one the BookLore container uses). BOOKLORE_DB_HOST defaults to booklore-mariadb; change it only if your DB container has a different name.
• BOOKLORE_LIBRARY_ID — which library to enrich. In BookLore go to Settings → Libraries; the id is usually shown there (or check the DB if unsure). The default 2 is the TTRPG library on the author's setup.
• BOOKLORE_NETWORK — the Docker network BookLore's database is attached to, so this tool can reach it. Run docker inspect booklore-mariadb and look under Networks, or check the networks: in BookLore's compose. Put that name in .env; the default is xen-server_default.

Your Komga details (KOMGA_URL, KOMGA_API_TOKEN) — needed for comics, and for sort to dedupe comics against Komga.

How do I find my Komga URL and API token?

KOMGA_URL is your Komga address as the container sees it — a LAN address like http://192.168.1.209:25600, not a public domain. For the token, in Komga open Account Settings → API Keys → Generate, and paste it into KOMGA_API_TOKEN.

---

Settings

All settings live in .env. Most have sensible defaults; you only must set the ones for the modes you run.

Variable	Default	What it does
ITCH_API_KEY	(none)	Your itch.io API key. Required for fetch.
STAGING_DIR	/staging	Working folder where fetch downloads, and sort/comics read from.
PARALLEL	4	How many itch downloads to run at once during fetch.
SORT_TTRPG_DIR	/books/TTRPG	Where sort files Physical game (TTRPG) books.
SORT_EBOOK_DIR	/books/eBooks	Where sort files Book (ebook) items.
SORT_COMICS_DIR	/books/Comics	Where sort files Comic items (Komga).
KOMGA_URL	http://192.168.1.209:25600	Your Komga address (LAN, not a public domain).
KOMGA_API_TOKEN	(none)	Komga API token, for comics and comic dedupe in sort.
BOOKLORE_DB_PASSWORD	(none)	BookLore's MariaDB password. Required for report/apply/watch/series.
BOOKLORE_DB_HOST	booklore-mariadb	BookLore's database container name.
BOOKLORE_DB_PORT	3306	BookLore's database port.
BOOKLORE_DB_USER	booklore	BookLore's database user.
BOOKLORE_DB_NAME	booklore	BookLore's database name.
BOOKLORE_LIBRARY_ID	2	Which BookLore library report enriches.
BOOKLORE_DATA_DIR	./booklore-data	Host path to BookLore's data dir, where apply writes cover files.
BOOKLORE_NETWORK	xen-server_default	The Docker network BookLore's DB is on, so this tool can reach it.
ENRICHER_POLITE_SECONDS	0.3	Pause between books when querying, to be polite to the source sites.
WATCH_LIBRARY_IDS	2	watch: comma-separated library ids to keep enriched (e.g. 1,2).
WATCH_BASELINE_EXISTING	1	watch: on first start, mark current books "seen" so only new arrivals get enriched.
WATCH_PATH	/books	watch: folder to monitor for new files.
WATCH_INTERVAL	300	watch: seconds between backstop polls (inotify wakes it sooner).
WATCH_MIN_CONFIDENCE	0.85	watch: only auto-apply matches this confident or better; rest are logged.
WATCH_DEBOUNCE_SECONDS	45	watch: wait this long after a file change before querying, so BookLore finishes importing.

Settings you can also pass as flags instead of env vars

Several settings have matching command-line flags if you'd rather override them per run: --commit (the write switch for sort/apply/comics/series), --library-id, --library-ids, --staging, --ttrpg-dir, --ebook-dir, --comics-dir, --data-dir, --outdir, --report, --parallel, --polite, --watch-path, --interval, --min-confidence, and --only-unenriched (report: skip books that already have a locked cover).

---

How matching & naming works

Every enrich match is scored on both title and author, comparing against your on-disk Author/Title folder layout, using two complementary sources:

• itch.io — public search plus each game page's OpenGraph tags; nails indie titles (FIST, Mörk Borg hacks, 200-word RPGs, jam games) that mainstream book providers don't have.
• DriveThruRPG — the open api.drivethrurpg.com JSON (the www host is Cloudflare-gated); covers published books with author, publisher, and rating.

Confidence tiers gate everything: HIGH is pre-approved to apply, MEDIUM is flagged for your review, LOW is skipped. apply writes only the rows you approved (the decision column), and locks every field it sets so a later BookLore refresh won't clobber it. Covers are written the BookLore-native way (data/images/<id>/cover.jpg plus a 250px thumbnail, with book_cover_hash bumped); Komga summaries go in via its API.

Why fetch needs two passes

itch's --filter-files-regex silently drops owned-game books, because their download URL has no file extension to match on. So fetch runs in two passes: first it classifies your whole library by itch category, then it re-fetches only the book games, unfiltered.

---

Why this exists

BookLore's built-in metadata providers (Google Books, Hardcover, RanobeDB…) have essentially zero coverage of indie/itch.io tabletop games and zines, and itch-dl on its own gives you an unsorted, metadata-less dump that mixes books in with video games. This pipeline covers the whole path — fetch → sort → enrich → keep-fresh — so a TTRPG library imported from, say, a Calibre export ends up with proper covers and blurbs instead of blank tiles.

itch.io library ──fetch──► staging ──sort──► TTRPG / ebook (BookLore) · Comics (Komga)
                                                    │                         │
                                          report→apply / watch            comics
                                          (itch + DriveThruRPG)        (itch → Komga)
                                          covers + blurbs + ratings

---

Building & publishing your own copy

For maintainers — build the image and push it to a registry

docker build -t itch-library-pipeline:latest .
# then tag/push to your registry, e.g. a self-hosted Forgejo:
docker tag itch-library-pipeline:latest your.forgejo.host/you/itch-library-pipeline:latest
docker push your.forgejo.host/you/itch-library-pipeline:latest

---

Good to know

• Nothing is destructive until --commit. report and dry-run sort/comics/series only write CSVs; fetch only writes to your staging folder. Always read the CSV before committing.
• Matching isn't perfect. Skim out/report.csv and check the MEDIUM-tier rows before you apply.
• Book downloads come courtesy of itch.io creators and the itch-dl tool; please be a good citizen — the pipeline paces its requests (ENRICHER_POLITE_SECONDS), so don't crank that to zero.

License

GNU GPL v3 (or later) — see LICENSE. Copyleft, so derivatives stay free software.