Load Fall 2026 courses into prod and relaunch #33

New issue

Open

opened 2026-10-03 19:44:37 +00:00 by jalenluorion · 0 comments

jalenluorion commented

2026-10-03 19:44:37 +00:00

Member

Goal
Classes are the base layer of the calendar. Prod needs the full Fall 2026 Schedule of Classes before we tell students to come back.

Current state

api/scraper/scripts/export_soc.py scrapes SOC and upserts orgs, courses, categories, events and recurrence rules, then asks the API to build occurrences (/api/events/regenerate_occurrences_by_events, batches of 50).
The semester is inferred (infer_semester_label in api/scraper/helpers/semester.py): run in October, it resolves to Fall_26. Reruns are safe (upserts only).
.forgejo/workflows/scrape.yml cannot do this yet: the OpenBao ci role only reads secret/secretspec/cal/ci/ (see the keepalive.yml header), so run it from a laptop.

Scope

Get the prod values from OpenBao: SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, API_BASE_URL, SCRAPER_API_TOKEN (resolve the prod profile with secretspec; never paste them in chat).
From api/: python -m scraper.scripts.export_soc --dry-run. Check the org/course/event counts and that duplicate identities is 0.
Real run: python -m scraper.scripts.export_soc (about 7.5 minutes). Every regeneration batch should report 0 failed; if not, rerun.
Spot check on cmucal.com: add a few courses to a schedule and compare lecture/recitation times with SIO.
Announce the relaunch (ScottyLabs channels, club/class group chats).

Done when

Fall 2026 courses show on prod calendars with correct times.
Run output (counts, failures) is pasted in a comment on this issue.
Relaunch announcement is posted.

Notes
For the tech lead. This makes a good live demo at a work session: dry run, real run, then watch the courses appear.
The scrape.yml header says a SOC run takes "roughly an hour"; that is stale, so update it while you are there.

**Goal** Classes are the base layer of the calendar. Prod needs the full Fall 2026 Schedule of Classes before we tell students to come back. **Current state** - `api/scraper/scripts/export_soc.py` scrapes SOC and upserts orgs, courses, categories, events and recurrence rules, then asks the API to build occurrences (`/api/events/regenerate_occurrences_by_events`, batches of 50). - The semester is inferred (`infer_semester_label` in `api/scraper/helpers/semester.py`): run in October, it resolves to `Fall_26`. Reruns are safe (upserts only). - `.forgejo/workflows/scrape.yml` cannot do this yet: the OpenBao `ci` role only reads `secret/secretspec/cal/ci/` (see the `keepalive.yml` header), so run it from a laptop. **Scope** - [ ] Get the prod values from OpenBao: `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, `API_BASE_URL`, `SCRAPER_API_TOKEN` (resolve the `prod` profile with secretspec; never paste them in chat). - [ ] From `api/`: `python -m scraper.scripts.export_soc --dry-run`. Check the org/course/event counts and that duplicate identities is 0. - [ ] Real run: `python -m scraper.scripts.export_soc` (about 7.5 minutes). Every regeneration batch should report 0 failed; if not, rerun. - [ ] Spot check on cmucal.com: add a few courses to a schedule and compare lecture/recitation times with SIO. - [ ] Announce the relaunch (ScottyLabs channels, club/class group chats). **Done when** - [ ] Fall 2026 courses show on prod calendars with correct times. - [ ] Run output (counts, failures) is pasted in a comment on this issue. - [ ] Relaunch announcement is posted. **Notes** For the tech lead. This makes a good live demo at a work session: dry run, real run, then watch the courses appear. The `scrape.yml` header says a SOC run takes "roughly an hour"; that is stale, so update it while you are there.

jalenluorion added this to the Phase 1 milestone

2026-10-03 19:44:37 +00:00

jalenluorion added the

chore

data

labels

2026-10-03 19:44:37 +00:00

jalenluorion added this to the CMUCal project

2026-10-03 19:45:09 +00:00