Evaluate new course and Stellic data files #202

New issue

Open

opened 2026-06-19 05:12:55 +00:00 by TenType · 0 comments

TenType commented

2026-06-19 05:12:55 +00:00

(Migrated from codeberg.org)

Summary

There's a LOT of data we have recently acquired that can improve our course commands and possibly let us add new ones! We need to take inventory of all this data, figure out what we can get out of this, and decide how to move forward.

Data Files

Dalmatian Data

Every file in src/data of this repository.

Courses Data

courses.json - up to F26
schedules.json - up to F26
syllabi.json - up to S25
geneds.json - up to F24
fces.json - up to S25 (note that we have an fces.csv already, so this is redundant)
professors.json - up to F21

Stellic-Scraped Data

There are three large folders. They each represent data gathered from a specific student in Stellic, where each folder has:

courses-history- lots of course data with descriptions, prereqs, locations, etc
programs - data on each school's program, along with specific gened/course requirements the student had access to

Objectives

Write up the following in a Discord message, comment, or document:

Provide a broad overview of what we have here and what this data gives us. If you find something new we haven't added to dalmatian yet (like course location history!), make note of that.
Identify the most recent semester in which each data file is accurate (like how it's done above for courses data)
Think about what's the best way to consolidate and store all this data in this repo, from a high-level perspective. Some guiding questions: What data attributes are in multiple files? What data is not useful to us? How do we want to format, store, and use this data in dalmatian moving forward?

Suggested Approach

This is a different issue from other ones in that you will write up what you have found. You don't need to edit any code in this repo, but you may find it helpful to write scripts that can parse the data and see what is there.

To get started, assign yourself here and message @tentype to get all the files!

## Summary There's a LOT of data we have recently acquired that can improve our course commands and possibly let us add new ones! We need to take inventory of all this data, figure out what we can get out of this, and decide how to move forward. ## Data Files ### Dalmatian Data Every file in `src/data` of this repository. ### Courses Data - `courses.json` - up to F26 - `schedules.json` - up to F26 - `syllabi.json` - up to S25 - `geneds.json` - up to F24 - `fces.json` - up to S25 (note that we have an `fces.csv` already, so this is redundant) - `professors.json` - up to F21 ### Stellic-Scraped Data There are three large folders. They each represent data gathered from a specific student in [Stellic](academicaudit.andrew.cmu.edu), where each folder has: - `courses-history`- lots of course data with descriptions, prereqs, locations, etc - `programs` - data on each school's program, along with specific gened/course requirements the student had access to ## Objectives Write up the following in a Discord message, comment, or document: - [ ] Provide a broad overview of what we have here and what this data gives us. If you find something new we haven't added to dalmatian yet (like course location history!), make note of that. - [ ] Identify the most recent semester in which each data file is accurate (like how it's done above for courses data) - [ ] Think about what's the best way to consolidate and store all this data in this repo, from a high-level perspective. Some guiding questions: What data attributes are in multiple files? What data is not useful to us? How do we want to format, store, and use this data in dalmatian moving forward? ## Suggested Approach This is a different issue from other ones in that you will write up what you have found. You don't need to edit any code in this repo, but you may find it helpful to write scripts that can parse the data and see what is there. To get started, assign yourself here and message @tentype to get all the files!