SOC export moves a course to a new org when its title changes #69

New issue

Open

opened 2026-10-03 19:44:43 +00:00 by jalenluorion · 0 comments

jalenluorion commented

2026-10-03 19:44:43 +00:00

Member

What's wrong
COURSE orgs are keyed by name, and the name includes the title: "15-112 <course title>" (soc_org_course.py L9-15). upsert_orgs() upserts on_conflict="name", and upsert_courses() sets courses.org_id to this run's org (upsert on course_number). When a title changes between semesters:

a new org is created, with new empty "Lectures"/"Recitations" categories and no admins;
courses.org_id moves to it, and so does course_agent (which reads courses.org_id in course_agent/app/db/repositories.py);
the old org keeps the admins, custom categories, earlier events and users' schedule entries.

Related: continuation rows append a subtitle to the name (monitors/academic/schedule_of_classes.py L179-181). In Fall 2026, 71 course numbers have more than one name in the same semester (e.g. "Digital Photography I" and "Digital Photography I TBA"). The org takes the first row's name and courses.course_name takes the last row's.

How to reproduce
On prod (read-only):

select split_part(name, ' ', 1) as num, count(*) from organizations
where type = 'COURSE' group by 1 having count(*) > 1;

Expected
One org per course number across semesters. A title change renames that org instead of creating a new one.

Where to look

api/scraper/transforms/soc_org_course.py, api/scraper/persistence/supabase_org_course.py (upsert_orgs() L4-34, upsert_courses() L37-81).
Look up the existing org through courses.org_id first; decide (with leads) how to merge existing duplicates.
Line numbers are from main at 2c5f2e9.

Done when

a rerun after a title change keeps the same org id
test in api/tests/scraper_monitors/test_upsert_orgs.py (it already has a fake Supabase table) for a renamed course

**What's wrong** COURSE orgs are keyed by name, and the name includes the title: `"15-112 <course title>"` (`soc_org_course.py` L9-15). `upsert_orgs()` upserts `on_conflict="name"`, and `upsert_courses()` sets `courses.org_id` to this run's org (upsert on `course_number`). When a title changes between semesters: - a new org is created, with new empty "Lectures"/"Recitations" categories and no admins; - `courses.org_id` moves to it, and so does `course_agent` (which reads `courses.org_id` in `course_agent/app/db/repositories.py`); - the old org keeps the admins, custom categories, earlier events and users' schedule entries. Related: continuation rows append a subtitle to the name (`monitors/academic/schedule_of_classes.py` L179-181). In Fall 2026, 71 course numbers have more than one name in the same semester (e.g. "Digital Photography I" and "Digital Photography I TBA"). The org takes the first row's name and `courses.course_name` takes the last row's. **How to reproduce** On prod (read-only): ``` select split_part(name, ' ', 1) as num, count(*) from organizations where type = 'COURSE' group by 1 having count(*) > 1; ``` **Expected** One org per course number across semesters. A title change renames that org instead of creating a new one. **Where to look** - `api/scraper/transforms/soc_org_course.py`, `api/scraper/persistence/supabase_org_course.py` (`upsert_orgs()` L4-34, `upsert_courses()` L37-81). - Look up the existing org through `courses.org_id` first; decide (with leads) how to merge existing duplicates. - Line numbers are from main at 2c5f2e9. **Done when** - [ ] a rerun after a title change keeps the same org id - [ ] test in `api/tests/scraper_monitors/test_upsert_orgs.py` (it already has a fake Supabase table) for a renamed course

jalenluorion added this to the Phase 2 milestone

2026-10-03 19:44:43 +00:00

jalenluorion added the

data

bug

intermediate

labels

2026-10-03 19:44:43 +00:00

jalenluorion added this to the CMUCal project

2026-10-03 19:45:09 +00:00