Commit graph

17 commits

Author SHA1 Message Date
rebuilt
4f035f6a63 feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page 2023-07-07 09:14:36 -07:00
Nelson Jovel
0a2c5e02c5 feat: add ability to merge disaggregation data with raw survey data to
produce a cleaned csv with merged income disaggregation columns
2023-06-20 12:22:24 -07:00
rebuilt
e2d24a9bec Fix: make sure values don't get reordered after copying over row values from survey item variants. This fixes a problem where cleaner would produce a row with likert scores that got shifted to align with the wrong column 2023-06-08 09:26:56 -07:00
rebuilt
ddf9a628d5 Fix: enable correct detection of student surveys types by rejecting any
headers ending with '-1' (the variants of standard questions)
2023-06-07 12:38:58 -07:00
rebuilt
abe7a8804c Don't check standard deviation for early education surveys 2023-06-02 16:05:45 -07:00
rebuilt
9aeb5f92af Missing progress or duration information does not result in a row removed in the cleaning process 2023-06-02 15:23:21 -07:00
rebuilt
e3ae12b425 update response_date to recorded_date 2023-05-31 16:57:47 -07:00
rebuilt
8ef8cfce58 Adjust valid duration threshold of short form items 2023-05-26 18:30:44 -07:00
rebuilt
4509c157fa Add automated data cleaning. Modify SurveyItemValues class to use regex
instead of hard coded values.  Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-16 13:38:29 -07:00
rebuilt
b250ebe415 Memoize schools in SurveyItemValues and academic_years in AcademicYear
for performace improvement
2023-03-28 03:38:52 -07:00
rebuilt
c15cb7b483 Change survey data loader spec to use factorybot objects instead of loading seeds. Change databasecleaner to use transaction. Add back babel-preset dependency to fix failing javascript test in production. 2023-03-28 03:06:17 -07:00
rebuilt
01851d8527 Make sure blank grades in csv load as nils instead of zeros 2023-02-10 13:44:48 -08:00
rebuilt
3a744db975 Start adding kindergarten 2023-01-17 15:27:15 -08:00
rebuilt
03c6bff7bb Load survey responses from sftp folder 2023-01-12 17:04:57 -08:00
rebuilt
eeeb4f5d14 Add Stoklosa survey responses 2023-01-12 17:02:35 -08:00
rebuilt
d5533bb756 Support survey item variants. Survey item id's in the format s-sbel-q1-1 will load into s-sbel-q1 2023-01-12 17:02:05 -08:00
rebuilt
a1b7db9b2e Add ability to load student demographic information for only lowell 2022-12-03 11:33:11 -08:00