From e4f0bc265ac3adb9f56b38b0a66477a289b66dfb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ebbe=20Ba=C3=9F?= Date: Wed, 9 Sep 2026 13:55:58 +0200 Subject: [PATCH] updated documentation --- README.md | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 02e210b..6dcb771 100644 --- a/README.md +++ b/README.md @@ -128,7 +128,7 @@ DATA_FILE=/path/to/events.json ARCHIVE_DATA_FILE=/path/to/archive.json uvicorn m | `GET` | `/events/{index}` | A single event by its 0-based position in the merged list; `404` if out of range | | `GET` | `/artists` | Deduplicated, case-insensitively sorted list of all artist names | | `GET` | `/calendar.ics` | iCalendar/webcal feed of all events — subscribe from any calendar app | -| `POST` | `/reload` | Re-read the data file(s) from disk (call after re-running the scraper); `500` if the main file is missing | +| `POST` | `/reload` | Re-read the data file(s) from disk (call after re-running the scraper); rate-limited (below); `500` if the main file is missing | ### `/events` query parameters @@ -165,16 +165,26 @@ API. `GET /health` reports, for both the regular and archive scrape: - `last_scrape_at` / `seconds_since_last_scrape` — from the output file's mtime, so this works no matter how the scrape was triggered. - `next_scrape_at` / `seconds_until_next_scrape` — computed from a cron - expression *you provide*, since the API can't reliably read another - process's crontab. Set it to match what's actually in cron: + expression, sourced in order: + 1. `SCRAPE_CRON` / `ARCHIVE_SCRAPE_CRON` env vars, if set — always wins, + and the only option that works when the API doesn't run as the same + user/host as the cron job. + 2. Otherwise, the API's own OS user's crontab (`crontab -l`), looked up + for a line invoking `scraper.py` (with vs. without `--archive` picks + archive vs. plain; an `@reboot` line for the same script is skipped + since it isn't a recurring schedule). Only works when the API process + runs as the same user whose personal crontab holds the scrape job — + not a system crontab/cron.d entry, not a job under a different user + or host. ```bash SCRAPE_CRON="0 * * * *" ARCHIVE_SCRAPE_CRON="0 4 * * *" uvicorn main:app ``` -`ARCHIVE_SCRAPE_CRON` falls back to `SCRAPE_CRON` if unset (handy if both -scrapes run off the same cron line). Leaving both unset just omits the -`next_scrape_at` fields (`null`) — `last_scrape_at` still works. +`ARCHIVE_SCRAPE_CRON` falls back to `SCRAPE_CRON` (env or crontab-discovered) +if unset (handy if both scrapes run off the same cron line). If neither an +env var nor a matching crontab line is found, you just get the last-scrape +info with `next_scrape_at` / `seconds_until_next_scrape` as `null`. ```json "scrape": { @@ -186,6 +196,17 @@ scrapes run off the same cron line). Leaving both unset just omits the } ``` +### `/reload` rate limiting + +`POST /reload` is limited to one call per `RELOAD_MIN_INTERVAL_SECONDS` +(default 10) — a call within that window returns `429` with a `Retry-After` +header instead of re-reading the file(s). It's a single shared cooldown, not +per-caller, so it also protects the server if several callers hit it at once. + +```bash +RELOAD_MIN_INTERVAL_SECONDS=30 uvicorn main:app +``` + ### Calendar / webcal feed `GET /calendar.ics` renders every loaded event (upcoming + archived) as an @@ -259,6 +280,8 @@ curl -X POST http://127.0.0.1:8000/reload │ ├── main.py # FastAPI app + routes │ ├── datasource.py # loads/merges/reloads the JSON files, date & artist helpers │ ├── calendar_feed.py # builds the /calendar.ics webcal feed +│ ├── scrape_schedule.py # last/next-scrape info for /health (env var or crontab) +│ ├── rate_limit.py # shared cooldown gate used by /reload │ ├── models.py # Pydantic models for the event schema │ ├── requirements.txt │ └── README.md