Nelson Jovel
6558ed491a
perf: reduce number of calls for student and teacher survey items in response rate calculators
2023-12-27 07:37:03 -08:00
Nelson Jovel
ec53cc9ae4
fix: correct a problem where survey items with insufficient responses were contributing to the response rate calculation
2023-12-26 22:14:56 -08:00
Nelson Jovel
c0b8c408ea
perf: reduce time spent in sql query for student response rate
2023-12-26 21:01:12 -08:00
Nelson Jovel
2f74153114
chore: refactor teacher response rate
2023-12-26 20:58:08 -08:00
Nelson Jovel
b0c9a0451e
perf: reduce number of queries for teacher survey_items
2023-12-26 17:32:39 -08:00
Nelson Jovel
8fb0a280ca
perf: reduce number of queries for respondents
2023-12-26 16:23:29 -08:00
Nelson Jovel
f026ea22ef
perf: reduce number of queries by requesting grouped_responses once instead of for each measure
2023-12-23 21:18:09 -08:00
Nelson Jovel
ed8e621333
chore: remove method that wasn't needed
2023-12-21 20:29:18 -08:00
Nelson Jovel
9e8f917457
chore: performance fixes for response rate calculation
2023-12-21 20:15:11 -08:00
Nelson Jovel
6a283b1fc8
chore: prevent multiple lookups for the boston district by inlining the
...
sql
2023-12-21 13:54:54 -08:00
Nelson Jovel
cc5e0b142c
chore: memoize grades for school
2023-12-21 13:44:07 -08:00
Nelson Jovel
60927e3271
fix: reduce number of n+1 queries
2023-12-21 13:34:16 -08:00
Nelson Jovel
ebf4ca4166
chore: move code that normalizes race data to the Race class
2023-12-20 19:07:16 -08:00
Nelson Jovel
56b512810c
chore: remove lowell seeding rules
2023-12-20 14:03:15 -08:00
Nelson Jovel
880b438eb4
chore: reenable test spec that tests data loader for races
2023-12-20 12:39:44 -08:00
Nelson Jovel
36e21515c3
chore: refactor Race out of survey_item_values
2023-12-20 12:25:23 -08:00
Nelson Jovel
e7fb009425
chore: refactor Gender out out of survey_item_values row
2023-12-20 11:08:23 -08:00
Nelson Jovel
f9717f80c7
chore: remove todo
2023-12-19 15:51:13 -08:00
Nelson Jovel
5df890923f
feat: don't show collection date of response rate if there is no survey data
2023-12-18 15:16:56 -08:00
Nelson Jovel
ea079e927b
chore: rename counts_by_grade to enrollment_by_grade for clarity
2023-12-18 15:01:33 -08:00
Nelson Jovel
a75373f6f2
feat: don't double count early ed students who are taking survey in parts
2023-12-18 14:57:15 -08:00
Nelson Jovel
934f2d7f30
fix: Correctly navigate the user to the latest academic year when no data exists for the school
2023-12-12 10:57:46 -08:00
Nelson Jovel
41d942c214
chore: Make sure 'hispanic' column only gets applied when using SIS race information
2023-12-12 10:53:07 -08:00
Nelson Jovel
24dbf33bcd
chore: remove unneeded comment
2023-12-11 15:55:00 -08:00
Nelson Jovel
f028e6c884
feat: if the filename includes the words 'form' or 'part' add that to the resulting cleaned filename
2023-12-11 15:39:20 -08:00
Nelson Jovel
d90a83e510
fix: instead of looking for 'asian' at the start of a word, look for it
...
after a word boundary. This means it still doesn't get confused with
caucasian and it's more flexible whan asian appears inside other text
such as 'Caucasian and Asian and Black'
2023-12-08 14:16:50 -08:00
Nelson Jovel
3f44613085
chore: various fixes for race and gender categorization during cleaning.
...
Also add tests for race and gender categorization
2023-12-08 13:12:19 -08:00
Nelson Jovel
883731bce1
feat: Support SIS categories for race in the form of 'White and Asian'
...
or 'Black, Asian, and white'
2023-12-07 13:40:02 -08:00
Nelson Jovel
0931950eaf
chore: make sure to output race and gender columns during cleaning
2023-12-07 13:39:25 -08:00
Nelson Jovel
3db0f9f757
fix: Print out message to make clean when there are duplicate headers
...
present in the raw survey file
2023-12-07 12:28:24 -08:00
Nelson Jovel
6541b87e9c
feat: add 2023-24 academic year and make sure previous year enrollment
...
and staffing data get loaded when missing
2023-12-07 12:27:19 -08:00
Nelson Jovel
e203be056a
chore: refactor code so we use a hash instead of a switch statement for long-term performance
2023-12-06 15:00:33 -08:00
Nelson Jovel
b7e670bb60
Lower threshold for the number of valid student responses from 17 to 11
2023-12-06 14:15:19 -08:00
Nelson Jovel
7dc881f654
chore: refactor code to put logic into models
2023-12-06 14:14:56 -08:00
Nelson Jovel
6e05909423
chore: fix categorization of gender
2023-12-06 14:12:27 -08:00
Nelson Jovel
6776ec8e2c
chore: add prefer not to disclose to gender. Add code of 7 as an option for prefer to self describe
2023-12-06 14:10:29 -08:00
Nelson Jovel
e325f38c43
Convert gender and race text into qualtrics codes during cleaning. Abide by 'prefer not to disclose' for self reported race. Give priority to self reported data but use SIS information as backup
2023-12-06 14:10:16 -08:00
Nelson Jovel
9efc1f41c6
chore: Add comment about StaffingLoader also cloning enrollment data when it clones staffing data
2023-12-06 14:09:54 -08:00
Nelson Jovel
10c052771d
Revert "chore: Add comment that notifies that cloning previous year staffing"
...
This reverts commit 95f8ada48d
2023-12-06 14:09:44 -08:00
Nelson Jovel
f677af826b
chore: Add comment that notifies that cloning previous year staffing
...
data will also copy over enrollment data
2023-12-06 14:09:33 -08:00
Nelson Jovel
f6f78bcd58
fix: make sure to grab the 'Gender- Qcode' column
2023-12-06 14:09:13 -08:00
Nelson Jovel
8bebe7db42
chore: since it's now possible for there to be multiple district and dese id columns separated by a dash and a number, be more explicit when we only want to filter out survey item ids that end in a -1
2023-11-06 14:52:21 -08:00
Nelson Jovel
b63c327d33
chore: when searching for dese id, split up pattern so that to be more explicit about the order in which to search out the columns that might have the dese ID we're looking for.
2023-11-06 13:15:50 -08:00
rebuilt
cddea60c8b
feat: reduce number of rows to process at one time to reduce memory use
2023-11-02 12:21:32 -07:00
rebuilt
1a707eb6bc
feat: load student responses in the same pass as loading the survey responses
...
chore: remove student loader since loading students is now done with the survey response loader
2023-11-02 09:52:39 -07:00
rebuilt
e3fbbabce5
feat: We no longer trust the progress number that gets exported from qualtrics. Instead during the cleaning progress, perform a manual count of the number of responses to filter out rows that don't meet the minimum threshold.
2023-10-27 15:12:24 -07:00
rebuilt
8baaa67730
fix: make sure sped is taken into account when caching analyze page
2023-10-11 11:15:13 -07:00
rebuilt
83661540b7
chore: upgrade to rails 7.1.
...
upgrade rspec
fix failing tests
upgrade devise
2023-10-11 10:58:52 -07:00
rebuilt
48e795fcfb
feat: add special education disaggregation
2023-10-06 11:41:52 -07:00
rebuilt
060d7aa55a
Add disaggregation by ELL
2023-09-29 19:29:23 -07:00
