Commit graph

14 commits

Author SHA1 Message Date
rebuilt
ddf9a628d5 Fix: enable correct detection of student surveys types by rejecting any
headers ending with '-1' (the variants of standard questions)
2023-06-07 12:38:58 -07:00
rebuilt
abe7a8804c Don't check standard deviation for early education surveys 2023-06-02 16:05:45 -07:00
rebuilt
9aeb5f92af Missing progress or duration information does not result in a row removed in the cleaning process 2023-06-02 15:23:21 -07:00
rebuilt
e3ae12b425 update response_date to recorded_date 2023-05-31 16:57:47 -07:00
rebuilt
8ef8cfce58 Adjust valid duration threshold of short form items 2023-05-26 18:30:44 -07:00
rebuilt
4509c157fa Add automated data cleaning. Modify SurveyItemValues class to use regex
instead of hard coded values.  Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-16 13:38:29 -07:00
rebuilt
b250ebe415 Memoize schools in SurveyItemValues and academic_years in AcademicYear
for performace improvement
2023-03-28 03:38:52 -07:00
rebuilt
c15cb7b483 Change survey data loader spec to use factorybot objects instead of loading seeds. Change databasecleaner to use transaction. Add back babel-preset dependency to fix failing javascript test in production. 2023-03-28 03:06:17 -07:00
rebuilt
01851d8527 Make sure blank grades in csv load as nils instead of zeros 2023-02-10 13:44:48 -08:00
rebuilt
3a744db975 Start adding kindergarten 2023-01-17 15:27:15 -08:00
rebuilt
03c6bff7bb Load survey responses from sftp folder 2023-01-12 17:04:57 -08:00
rebuilt
eeeb4f5d14 Add Stoklosa survey responses 2023-01-12 17:02:35 -08:00
rebuilt
d5533bb756 Support survey item variants. Survey item id's in the format s-sbel-q1-1 will load into s-sbel-q1 2023-01-12 17:02:05 -08:00
rebuilt
a1b7db9b2e Add ability to load student demographic information for only lowell 2022-12-03 11:33:11 -08:00