fundi-scraper-api/api

Fundi Scraper API

A small REST API (FastAPI) that serves the events scraped by scraper/scraper.py from a JSON file.

Setup

cd api
pip install -r requirements.txt

Run

cd api
uvicorn main:app --reload

Interactive docs: http://127.0.0.1:8000/docs

By default the API reads ../fundi-scraped-output.json (upcoming events) and ../fundi-archive-output.json (past events, optional — produced by scraper.py --archive), merging both into one in-memory list. Point either elsewhere with an env var:

DATA_FILE=/path/to/events.json ARCHIVE_DATA_FILE=/path/to/archive.json uvicorn main:app

Scrape schedule reporting

/health reports when each scraper output file was last written (its mtime) and, if it can determine the cron schedule, when it's next due. The schedule is sourced in order:

1. SCRAPE_CRON / ARCHIVE_SCRAPE_CRON env vars, if set — always wins, and the only option that works when the API doesn't run as the same user/host as the cron job.
2. Otherwise, the API's own OS user's crontab (crontab -l), looked up for a line invoking scraper.py (with vs. without --archive picks archive vs. plain). Only works when the API process runs as the same user whose personal crontab holds the scrape job — not a system crontab/cron.d entry, not a job scheduled under a different user or host.

SCRAPE_CRON="0 * * * *" ARCHIVE_SCRAPE_CRON="0 4 * * *" uvicorn main:app

SCRAPE_CRON covers the plain scrape (fundi-scraped-output.json); ARCHIVE_SCRAPE_CRON covers --archive (fundi-archive-output.json) and falls back to SCRAPE_CRON (env or crontab-discovered) if unset — set it separately only if the archive scrape runs on its own cron line. If neither an env var nor a matching crontab line is found, you just get the last-scrape info with next_scrape_at / seconds_until_next_scrape as null.

/reload rate limiting

POST /reload is limited to one call per RELOAD_MIN_INTERVAL_SECONDS (default 10) - a call within that window returns 429 with a Retry-After header instead of re-reading the file(s). This is a single shared cooldown, not per-caller, so it also protects the server if several callers hit it at once.

RELOAD_MIN_INTERVAL_SECONDS=30 uvicorn main:app

Endpoints

Method	Path	Description
GET	/health	Status, event counts, and last/next scrape timing (see below)
GET	/events	List events, with optional filters (see below)
GET	/events/archive	Past events only — shorthand for /events?upcoming=false
GET	/events/{index}	Single event by its position in the merged list (0-based)
GET	/artists	Deduplicated, sorted list of all artists with every date they appear on
GET	/calendar.ics	iCalendar/webcal feed of all events — subscribe with webcal://<host>/calendar.ics
POST	/reload	Re-read the data file(s) from disk (after a fresh scrape) - rate-limited, see below

/events query parameters

/events/archive accepts the same parameters except upcoming (always false).

Param	Type	Meaning
free	bool	Only free / only paid events
name	string	Case-insensitive substring match on the event name
artist	string	Case-insensitive substring match on any artist name
date	string	Exact match on the raw date string (dd.mm.yy)
upcoming	bool	true = today or later, false = past events
sort	string	asc/desc = chronological by event date (unparseable dates sort last either way); az/za = alphabetical by event name (case-insensitive)
count	int ≥ 1	Limit the number of results returned (applied after filtering and sorting)

Examples:

curl 'http://127.0.0.1:8000/events?free=true'
curl 'http://127.0.0.1:8000/events?artist=randali&upcoming=true'
curl 'http://127.0.0.1:8000/events?sort=asc'
curl 'http://127.0.0.1:8000/events?sort=za'
curl 'http://127.0.0.1:8000/events?sort=asc&count=5'
curl 'http://127.0.0.1:8000/events/0'

/artists

Each artist appears once, with every date (past or upcoming) they're on the line-up for:

[
  { "artist_name": "Bizarre", "dates": ["05.09.26", "12.10.26"] },
  { "artist_name": "Skkin Velvet", "dates": ["04.09.26"] }
]

Param	Type	Meaning
sort	string	az/za = alphabetical by artist name (case-insensitive, default az); asc/desc = chronological by each artist's latest date (an artist has many dates, so this is the one used to place them) — artists with no parseable date sort last either way
count	int ≥ 1	Limit the number of artists returned

Each artist's own dates list is always chronological (earliest first), regardless of sort — sort only controls the order artists appear in.

curl 'http://127.0.0.1:8000/artists?sort=desc'          # most recently active first
curl 'http://127.0.0.1:8000/artists?sort=desc&count=10' # top 10 most recently active