changed method of fetching crontab schedule
This commit is contained in:
+14
-4
@@ -31,8 +31,17 @@ DATA_FILE=/path/to/events.json ARCHIVE_DATA_FILE=/path/to/archive.json uvicorn m
|
||||
### Scrape schedule reporting
|
||||
|
||||
`/health` reports when each scraper output file was last written (its mtime)
|
||||
and, if you tell it the cron schedule, when it's next due. This doesn't read
|
||||
your crontab — set the same expression(s) you put there as env vars:
|
||||
and, if it can determine the cron schedule, when it's next due. The schedule
|
||||
is sourced in order:
|
||||
|
||||
1. `SCRAPE_CRON` / `ARCHIVE_SCRAPE_CRON` env vars, if set — always wins, and
|
||||
the only option that works when the API doesn't run as the same user/host
|
||||
as the cron job.
|
||||
2. Otherwise, the API's own OS user's crontab (`crontab -l`), looked up for a
|
||||
line invoking `scraper.py` (with vs. without `--archive` picks archive vs.
|
||||
plain). Only works when the API process runs as the same user whose
|
||||
personal crontab holds the scrape job — not a system crontab/cron.d entry,
|
||||
not a job scheduled under a different user or host.
|
||||
|
||||
```bash
|
||||
SCRAPE_CRON="0 * * * *" ARCHIVE_SCRAPE_CRON="0 4 * * *" uvicorn main:app
|
||||
@@ -40,8 +49,9 @@ SCRAPE_CRON="0 * * * *" ARCHIVE_SCRAPE_CRON="0 4 * * *" uvicorn main:app
|
||||
|
||||
`SCRAPE_CRON` covers the plain scrape (`fundi-scraped-output.json`);
|
||||
`ARCHIVE_SCRAPE_CRON` covers `--archive` (`fundi-archive-output.json`) and
|
||||
falls back to `SCRAPE_CRON` if unset — set it separately only if the archive
|
||||
scrape runs on its own cron line. Leave both unset to just get the
|
||||
falls back to `SCRAPE_CRON` (env or crontab-discovered) if unset — set it
|
||||
separately only if the archive scrape runs on its own cron line. If neither
|
||||
an env var nor a matching crontab line is found, you just get the
|
||||
last-scrape info with `next_scrape_at` / `seconds_until_next_scrape` as
|
||||
`null`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user