b1f942133b
update parsing rules from glossary
2 years ago
ea2feb138b
add disaggregation glossary
2 years ago
Gabe Farrell
2f1cf8b9fd
Chore: Reduce queries
2 years ago
e60a30911c
chore: add more rules for finding disaggregation columns and parsing sped values
2 years ago
b992a9412b
chore: add new rules for sped parsing
2 years ago
8bb6f5e8f0
Add ell income and sped parsing rules for SIS data. Add tests for the
...
new inputs.
2 years ago
Gabe Farrell
b257f9dff7
Fix: Lowell -> Amherst
2 years ago
Gabe Farrell
40a35fff2d
Bug Fix: survey item by grade report
2 years ago
33da0859b9
Split academic year into seasons if the academic year's range is
...
initialized with a season, i.e. "2024-25 Fall". Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly. Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year. So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring". Add tests
for the new range format. Set the default cutoff for the start of Spring season will be the last Sunday in February
2 years ago
Gabe Farrell
5789ebf564
Faster admin data loader + rename School.school_hash
2 years ago
Gabe Farrell
963d945eea
Feat: Add item-level reporting by grade
2 years ago
Gabe Farrell
04d61c0b1a
Memoize admin data loader
2 years ago
Gabe Farrell
37766e777d
Fix: Isolate Report::SurveyItem
2 years ago
Gabe Farrell
b08214fe10
rename SurveyItemResponse method for clarity
2 years ago
Gabe Farrell
a085afb3b6
Feat: Add task to generate item-level survey report
2 years ago
65d3733189
feat: ignore library survey items in csvs
2 years ago
8a702efa51
fix: make sure staffing checks for zero
2 years ago
ad23bb1816
fix:typo
2 years ago
3106768175
fix: If all respondents have total_teachers set to 0, that will trigger cloning previous year data
2 years ago
90363d8b86
fix: make sure we are only reseeding staffing and demographic numbers if the data for that year is empty
2 years ago
51e71a7f83
fix: Fixes #187294757 . Data is hidden when selecting multiple years if the default year has no student data to show
2 years ago
0864b6e901
fix: make sure averages for admin data scales are included in report
2 years ago
e83865ad55
feat: create beyond learning loss report
2 years ago
7a4eb8302b
feat: change the text of the prompts for survey items
2 years ago
f15ea64dec
fix: Correctly namespace models to avoid crashing the build process
2 years ago
ee4343383c
feat: create a score csv report for measures
2 years ago
4c017cf857
fix: add pre-k to list of grades
2 years ago
2cb43058a0
fix: loosen restrictions on date format
2 years ago
289b04bc69
match an additional format for Dates. Supported dates are now '1/10/2022 14:21:45' '2022-1-10T14:21:45' '2022-1-10 14:21:45'
2 years ago
d6735d449d
feat: Support two date formats: ISO 8601 and the standard US date format
...
used in google sheets
2 years ago
03349b0ed5
chore: support alternate format for recorded date
2 years ago
37610e732f
chore: alphabetize districts when writing filename for cleaned files
2 years ago
b6de3349d6
fix: support alternate format for active admin data items. Support both 'True/False' and '1/0'
2 years ago
0a32fb50ff
fix: no longer support 'form' in filename when cleaning. Only look for 'part X' and add that to the filename if it exists
2 years ago
725bef2a8f
fix: Support true/false column values when categorizing income status. True is categorized as Economically Disadvantaged. False is categorized as Not Economically Disadvantaged
2 years ago
48e1f941f5
fix: change the default to not an english language learner. Blank columns are treated as Not ELL. Only mark student as an english language learner if it's marked as such. Columns marked with 'NA' are treated as unknown
2 years ago
3c210e0ab6
chore: rename admin data to school data on analyze page
2 years ago
90d1419d9c
feat: change 'school admin data' to 'school data'
2 years ago
6558ed491a
perf: reduce number of calls for student and teacher survey items in response rate calculators
2 years ago
ec53cc9ae4
fix: correct a problem where survey items with insufficient responses were contributing to the response rate calculation
2 years ago
c0b8c408ea
perf: reduce time spent in sql query for student response rate
2 years ago
2f74153114
chore: refactor teacher response rate
2 years ago
b0c9a0451e
perf: reduce number of queries for teacher survey_items
2 years ago
8fb0a280ca
perf: reduce number of queries for respondents
2 years ago
f026ea22ef
perf: reduce number of queries by requesting grouped_responses once instead of for each measure
2 years ago
ed8e621333
chore: remove method that wasn't needed
2 years ago
9e8f917457
chore: performance fixes for response rate calculation
2 years ago
6a283b1fc8
chore: prevent multiple lookups for the boston district by inlining the
...
sql
2 years ago
cc5e0b142c
chore: memoize grades for school
2 years ago
60927e3271
fix: reduce number of n+1 queries
2 years ago
ebf4ca4166
chore: move code that normalizes race data to the Race class
2 years ago
56b512810c
chore: remove lowell seeding rules
2 years ago
880b438eb4
chore: reenable test spec that tests data loader for races
2 years ago
36e21515c3
chore: refactor Race out of survey_item_values
2 years ago
e7fb009425
chore: refactor Gender out out of survey_item_values row
2 years ago
f9717f80c7
chore: remove todo
2 years ago
5df890923f
feat: don't show collection date of response rate if there is no survey data
2 years ago
ea079e927b
chore: rename counts_by_grade to enrollment_by_grade for clarity
2 years ago
a75373f6f2
feat: don't double count early ed students who are taking survey in parts
2 years ago
934f2d7f30
fix: Correctly navigate the user to the latest academic year when no data exists for the school
2 years ago
41d942c214
chore: Make sure 'hispanic' column only gets applied when using SIS race information
2 years ago
24dbf33bcd
chore: remove unneeded comment
2 years ago
f028e6c884
feat: if the filename includes the words 'form' or 'part' add that to the resulting cleaned filename
2 years ago
d90a83e510
fix: instead of looking for 'asian' at the start of a word, look for it
...
after a word boundary. This means it still doesn't get confused with
caucasian and it's more flexible whan asian appears inside other text
such as 'Caucasian and Asian and Black'
2 years ago
3f44613085
chore: various fixes for race and gender categorization during cleaning.
...
Also add tests for race and gender categorization
2 years ago
883731bce1
feat: Support SIS categories for race in the form of 'White and Asian'
...
or 'Black, Asian, and white'
2 years ago
0931950eaf
chore: make sure to output race and gender columns during cleaning
2 years ago
3db0f9f757
fix: Print out message to make clean when there are duplicate headers
...
present in the raw survey file
2 years ago
6541b87e9c
feat: add 2023-24 academic year and make sure previous year enrollment
...
and staffing data get loaded when missing
2 years ago
e203be056a
chore: refactor code so we use a hash instead of a switch statement for long-term performance
2 years ago
b7e670bb60
Lower threshold for the number of valid student responses from 17 to 11
2 years ago
7dc881f654
chore: refactor code to put logic into models
2 years ago
6e05909423
chore: fix categorization of gender
2 years ago
6776ec8e2c
chore: add prefer not to disclose to gender. Add code of 7 as an option for prefer to self describe
2 years ago
e325f38c43
Convert gender and race text into qualtrics codes during cleaning. Abide by 'prefer not to disclose' for self reported race. Give priority to self reported data but use SIS information as backup
2 years ago
9efc1f41c6
chore: Add comment about StaffingLoader also cloning enrollment data when it clones staffing data
2 years ago
10c052771d
Revert "chore: Add comment that notifies that cloning previous year staffing"
...
This reverts commit 95f8ada48d
2 years ago
f677af826b
chore: Add comment that notifies that cloning previous year staffing
...
data will also copy over enrollment data
2 years ago
f6f78bcd58
fix: make sure to grab the 'Gender- Qcode' column
2 years ago
8bebe7db42
chore: since it's now possible for there to be multiple district and dese id columns separated by a dash and a number, be more explicit when we only want to filter out survey item ids that end in a -1
2 years ago
b63c327d33
chore: when searching for dese id, split up pattern so that to be more explicit about the order in which to search out the columns that might have the dese ID we're looking for.
2 years ago
cddea60c8b
feat: reduce number of rows to process at one time to reduce memory use
2 years ago
1a707eb6bc
feat: load student responses in the same pass as loading the survey responses
...
chore: remove student loader since loading students is now done with the survey response loader
2 years ago
e3fbbabce5
feat: We no longer trust the progress number that gets exported from qualtrics. Instead during the cleaning progress, perform a manual count of the number of responses to filter out rows that don't meet the minimum threshold.
2 years ago
8baaa67730
fix: make sure sped is taken into account when caching analyze page
2 years ago
83661540b7
chore: upgrade to rails 7.1.
...
upgrade rspec
fix failing tests
upgrade devise
2 years ago
48e795fcfb
feat: add special education disaggregation
2 years ago
060d7aa55a
Add disaggregation by ELL
2 years ago
8d33095a48
fix: fix crash when viewing income graph on analyze page
2 years ago
aaa43cdd64
fix: I broke the feature where the site would automatically navigate the user to the latest year that had sufficient data when I made response rates calculate on each page load instead of being precalculated and stored in the database. Instead of a database lookup for response rates that meet the sufficiency threshold, I caculate the latest year when a user chooses a school from the welcome page.
2 years ago
abea2cb8fa
feat: support multiple columns for race and gender information
2 years ago
463e4c9452
fix: hide scores on analyze page for scores that don't meet the student threshold of 25%
2 years ago
714b90b3eb
fix: ensure cleaner outputs columns for all survey items. Before the fix, if a survey item varient (ending in -1, ie s-tint-q1-1) did not have a matching survey item s-tint-q1, the resulting csv would not include that column
2 years ago
2321897283
fix: start fixing problem with variants not getting added to the cleaned csv
2 years ago
315b398a5a
fix: Overall response rate was incorrectly using the updated_at date instead of the recorded date. Also, it was just using the last date for all academic years instead of the last date the survey was taken per academic year.
2 years ago
Gabe Farrell
a8db6c5ecf
Add Overall Response Rate Popover
2 years ago
b3efbdeb33
The overall response rate is artifically lower because we are including the number of all the students at the school, not just the number of students that took the survey. Updated the overall response rate presenter to return the count of only the grades that took the student survey.
2 years ago
65ca0fd1ac
Move color definitions out of response rate presenter and into _colors.scss
2 years ago
a785c69c44
Add Overall Response Rate
2 years ago
4afa030141
chore: remove precalculated race scores. Calculate race scores on every reload
2 years ago
Nelson Jovel
Gabe Farrell