fundi-scraper-api/api/README.md

Fundi Scraper API

A small REST API (FastAPI) that serves the events scraped by scraper/scraper.py from a JSON file.

Setup

cd api
pip install -r requirements.txt

Run

cd api
uvicorn main:app --reload

Interactive docs: http://127.0.0.1:8000/docs

By default the API reads ../fundi-scraped-output.json (upcoming events) and ../fundi-archive-output.json (past events, optional — produced by scraper.py --archive), merging both into one in-memory list. Point either elsewhere with an env var:

DATA_FILE=/path/to/events.json ARCHIVE_DATA_FILE=/path/to/archive.json uvicorn main:app

Scrape schedule reporting

/health reports when each scraper output file was last written (its mtime) and, if it can determine the cron schedule, when it's next due. The schedule is sourced in order:

1. SCRAPE_CRON / ARCHIVE_SCRAPE_CRON env vars, if set — always wins, and the only option that works when the API doesn't run as the same user/host as the cron job.
2. Otherwise, the API's own OS user's crontab (crontab -l), looked up for a line invoking scraper.py (with vs. without --archive picks archive vs. plain). Only works when the API process runs as the same user whose personal crontab holds the scrape job — not a system crontab/cron.d entry, not a job scheduled under a different user or host.

SCRAPE_CRON="0 * * * *" ARCHIVE_SCRAPE_CRON="0 4 * * *" uvicorn main:app

SCRAPE_CRON covers the plain scrape (fundi-scraped-output.json); ARCHIVE_SCRAPE_CRON covers --archive (fundi-archive-output.json) and falls back to SCRAPE_CRON (env or crontab-discovered) if unset — set it separately only if the archive scrape runs on its own cron line. If neither an env var nor a matching crontab line is found, you just get the last-scrape info with next_scrape_at / seconds_until_next_scrape as null.

Endpoints

Method	Path	Description
GET	/health	Status, event counts, and last/next scrape timing (see below)
GET	/events	List events, with optional filters (see below)
GET	/events/archive	Past events only — shorthand for /events?upcoming=false
GET	/events/{index}	Single event by its position in the merged list (0-based)
GET	/artists	Deduplicated, sorted list of all artist names
GET	/calendar.ics	iCalendar/webcal feed of all events — subscribe with webcal://<host>/calendar.ics
POST	/reload	Re-read the data file(s) from disk (after a fresh scrape)

/events query parameters

Param	Type	Meaning
free	bool	Only free / only paid events
name	string	Case-insensitive substring match on the event name
artist	string	Case-insensitive substring match on any artist name
date	string	Exact match on the raw date string (dd.mm.yy)
upcoming	bool	true = today or later, false = past events

Examples:

curl 'http://127.0.0.1:8000/events?free=true'
curl 'http://127.0.0.1:8000/events?artist=randali&upcoming=true'
curl 'http://127.0.0.1:8000/events/0'