# Fundi Scraper API A small REST API (FastAPI) that serves the events scraped by [`scraper/scraper.py`](../scraper/scraper.py) from a JSON file. ## Setup ```bash cd api pip install -r requirements.txt ``` ## Run ```bash cd api uvicorn main:app --reload ``` Interactive docs: http://127.0.0.1:8000/docs By default the API reads `../fundi-scraped-output.json` (upcoming events) and `../fundi-archive-output.json` (past events, optional — produced by `scraper.py --archive`), merging both into one in-memory list. Point either elsewhere with an env var: ```bash DATA_FILE=/path/to/events.json ARCHIVE_DATA_FILE=/path/to/archive.json uvicorn main:app ``` ### Scrape schedule reporting `/health` reports when each scraper output file was last written (its mtime) and, if it can determine the cron schedule, when it's next due. The schedule is sourced in order: 1. `SCRAPE_CRON` / `ARCHIVE_SCRAPE_CRON` env vars, if set — always wins, and the only option that works when the API doesn't run as the same user/host as the cron job. 2. Otherwise, the API's own OS user's crontab (`crontab -l`), looked up for a line invoking `scraper.py` (with vs. without `--archive` picks archive vs. plain). Only works when the API process runs as the same user whose personal crontab holds the scrape job — not a system crontab/cron.d entry, not a job scheduled under a different user or host. ```bash SCRAPE_CRON="0 * * * *" ARCHIVE_SCRAPE_CRON="0 4 * * *" uvicorn main:app ``` `SCRAPE_CRON` covers the plain scrape (`fundi-scraped-output.json`); `ARCHIVE_SCRAPE_CRON` covers `--archive` (`fundi-archive-output.json`) and falls back to `SCRAPE_CRON` (env or crontab-discovered) if unset — set it separately only if the archive scrape runs on its own cron line. If neither an env var nor a matching crontab line is found, you just get the last-scrape info with `next_scrape_at` / `seconds_until_next_scrape` as `null`. ### `/reload` rate limiting `POST /reload` is limited to one call per `RELOAD_MIN_INTERVAL_SECONDS` (default 10) - a call within that window returns `429` with a `Retry-After` header instead of re-reading the file(s). This is a single shared cooldown, not per-caller, so it also protects the server if several callers hit it at once. ```bash RELOAD_MIN_INTERVAL_SECONDS=30 uvicorn main:app ``` ## Endpoints | Method | Path | Description | |--------|-------------------|-------------| | GET | `/health` | Status, event counts, and last/next scrape timing (see below) | | GET | `/events` | List events, with optional filters (see below) | | GET | `/events/archive` | Past events only — shorthand for `/events?upcoming=false` | | GET | `/events/{index}` | Single event by its position in the merged list (0-based) | | GET | `/artists` | Deduplicated, sorted list of all artist names | | GET | `/calendar.ics` | iCalendar/webcal feed of all events — subscribe with `webcal:///calendar.ics` | | POST | `/reload` | Re-read the data file(s) from disk (after a fresh scrape) - rate-limited, see below | ### `/events` query parameters | Param | Type | Meaning | |------------|--------|---------| | `free` | bool | Only free / only paid events | | `name` | string | Case-insensitive substring match on the event name | | `artist` | string | Case-insensitive substring match on any artist name | | `date` | string | Exact match on the raw date string (`dd.mm.yy`) | | `upcoming` | bool | `true` = today or later, `false` = past events | Examples: ```bash curl 'http://127.0.0.1:8000/events?free=true' curl 'http://127.0.0.1:8000/events?artist=randali&upcoming=true' curl 'http://127.0.0.1:8000/events/0' ```