rebuilt
b6675c2516
ECP-206 Delete the 3 info boxes for each sub-category on the Browse page
2025-10-30 13:32:39 -07:00
rebuilt
6b75af4c7d
fix: make survey data uploads threaded
2025-09-17 21:57:34 -07:00
rebuilt
23e5f9bb48
fix: don't concat nils into array
2025-09-17 20:19:30 -07:00
rebuilt
65acd68d84
fix: update how survey data is loaded because some data was missing from previous uploads
2025-09-17 19:22:31 -07:00
rebuilt
138c61e7f0
ECP-201 Add an unknown categories to the parent survey SES filter
2025-07-09 21:54:45 -07:00
rebuilt
690f2db863
ECP-162 fix benefits so it gets scored correctly
2025-07-03 17:45:45 -07:00
rebuilt
a3082037fa
fix: ensure loaded survey item responses are uniq by id to avoid PG::CardinalityViolation: ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time (PG::CardinalityViolation)
2025-07-03 08:10:42 -07:00
rebuilt
1da1b238e0
fix: PG::CardinalityViolation: ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time (PG::CardinalityViolation) HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
2025-07-02 19:24:29 -07:00
rebuilt
cab0a3955e
finish up adding socio economic disag filter
2025-06-25 17:12:20 -07:00
rebuilt
a258b32b39
WIP: create backend for socio-economic-status
2025-06-25 11:08:12 -07:00
rebuilt
9d8543afae
feat: Update cleaner and uploader to load gender data from parent surveys
...
Implement graph views for parent gender data
2025-05-27 11:13:00 -07:00
rebuilt
5b00454a1b
ECP-125 feat:
...
Update cleaner to read parent races.
Update uploader to set races for parents.
Add race graphs to analyze page. Show both measure and scale level graphs
2025-05-22 11:57:26 -07:00
rebuilt
76f2467b97
chore: ECP-152 Reduce batch size of survey uploads to 100 at a time from 500 to reduce memory usage and also explicitly run garbage collection so we stay under the 500 mb limit of heroku
2025-05-12 19:15:33 -07:00
rebuilt
0f457becf0
feat: create a parents by language graph
...
Update demographics table with lanugage options
Create a lanugage table to hold the new languages
Update the demographic loader to input languages into the database
Update the cleaner to read the language column
Update the parent table to hold a reference to a language
Update the data uploader script to read the language from the csv and update the language information for any parent items that already exist (or create database entries if none already exist)
update the analyze interface to add controls for selecting ‘parents by group’ and a dropdown for ‘parent by language’
Update the analyze controller to read the parent-by-group parameter
Create a graph for the parent-by-group view
Bubble up averages for language calculations.
Make sure n-size only counts responses for a given measure.
2025-04-28 16:42:11 -07:00
Nelson Jovel
beb93aa8e8
fix: make sure all likert scores get counted even when the survey item id has different capitalization. Add tests for uploading parent data. Change the parent response rate calcuation to count all students in the school instead of just for the grades that were given the student survey
2024-11-08 15:36:34 -08:00
Nelson Jovel
0c661930e9
Show parent response rate
2024-11-03 12:47:25 -08:00
Nelson Jovel
e5e969b968
feat: add parent survey gauges
2024-09-16 15:24:43 -07:00
Nelson Jovel
532ca39e84
chore: speed up survey data loader
2024-05-23 12:23:52 -07:00
Nelson Jovel
33da0859b9
Split academic year into seasons if the academic year's range is
...
initialized with a season, i.e. "2024-25 Fall". Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly. Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year. So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring". Add tests
for the new range format. Set the default cutoff for the start of Spring season will be the last Sunday in February
2024-04-25 09:21:04 -07:00
5789ebf564
Faster admin data loader + rename School.school_hash
2024-04-22 14:46:37 -04:00
Nelson Jovel
56b512810c
chore: remove lowell seeding rules
2023-12-20 14:03:15 -08:00
Nelson Jovel
880b438eb4
chore: reenable test spec that tests data loader for races
2023-12-20 12:39:44 -08:00
Nelson Jovel
36e21515c3
chore: refactor Race out of survey_item_values
2023-12-20 12:25:23 -08:00
Nelson Jovel
e7fb009425
chore: refactor Gender out out of survey_item_values row
2023-12-20 11:08:23 -08:00
rebuilt
cddea60c8b
feat: reduce number of rows to process at one time to reduce memory use
2023-11-02 12:21:32 -07:00
rebuilt
1a707eb6bc
feat: load student responses in the same pass as loading the survey responses
...
chore: remove student loader since loading students is now done with the survey response loader
2023-11-02 09:52:39 -07:00
rebuilt
48e795fcfb
feat: add special education disaggregation
2023-10-06 11:41:52 -07:00
rebuilt
060d7aa55a
Add disaggregation by ELL
2023-09-29 19:29:23 -07:00
rebuilt
a785c69c44
Add Overall Response Rate
2023-08-09 15:13:58 -07:00
rebuilt
4f035f6a63
feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page
2023-07-07 09:14:36 -07:00
rebuilt
d72f8d31e0
fix: There was an n+1 problem where we looked up the list of schools for
...
every row. Now we query the list of schools just once per file
2023-06-26 11:36:03 -07:00
rebuilt
e8aa75bf66
feat: update survey_item_response table to indlude recorded date and import recorded date when loading responses
2023-06-23 11:26:53 -07:00
rebuilt
76b79b99c2
Fix: Parse headers when they are surrounded by quotes. This helps load recent csv files correctly
2023-06-06 18:18:52 -07:00
rebuilt
d6b2521883
Fix regression in student loader
2023-05-19 13:48:16 -07:00
rebuilt
4509c157fa
Add automated data cleaning. Modify SurveyItemValues class to use regex
...
instead of hard coded values. Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-16 13:38:29 -07:00
rebuilt
c15cb7b483
Change survey data loader spec to use factorybot objects instead of loading seeds. Change databasecleaner to use transaction. Add back babel-preset dependency to fix failing javascript test in production.
2023-03-28 03:06:17 -07:00
rebuilt
c0332955f3
move csv require statement to application.rb
2023-02-27 15:47:20 -08:00
rebuilt
c6b8518cb1
Fix bug with not all survey responses loading when using sftp loader
2023-01-12 17:05:00 -08:00
rebuilt
7a159f7ddf
Process 1000 rows at a time to limit memory usage in production
2023-01-12 17:05:00 -08:00
rebuilt
03c6bff7bb
Load survey responses from sftp folder
2023-01-12 17:04:57 -08:00
rebuilt
a1b7db9b2e
Add ability to load student demographic information for only lowell
2022-12-03 11:33:11 -08:00
rebuilt
fc9e73f656
make sure survey responses for Butler get loaded correctly
2022-12-02 14:11:54 -08:00
rebuilt
9bbba64e9e
Add rules to the survey data loader to allow skipping non-lowell schools
2022-12-01 16:15:45 -08:00
Nelson Jovel
43bc55ed32
Query for survey items once instead of on each row
2022-11-28 11:58:58 -08:00
rebuilt
83fa255e5d
Make utility method private
2022-11-25 03:50:17 -08:00
rebuilt
3ca270e3b1
Fix more malformed csv errors
2022-11-25 03:09:00 -08:00
rebuilt
360579852c
Send fewer sql queries when loading survey responses
2022-11-24 03:14:20 -08:00
rebuilt
9ae94a2dae
further reduce batch size to avoid exceeding memory quota
2022-11-23 13:54:11 -08:00
rebuilt
bb472ce6c3
Reduce number of rows to process to avoid exceeding memory limit on heroku
2022-11-23 12:32:31 -08:00
rebuilt
7d7bee6498
Add gender disagreggation
2022-10-27 15:56:50 -07:00
