d338003886
Show parent response rate
1 year ago
c85ddddd8d
feat: add parent survey gauges
1 year ago
8d31faf012
chore: speed up survey data loader
2 years ago
ac5c1edfbf
feat: Split academic year into seasons if the academic year's range is
...
initialized with a season, i.e. "2024-25 Fall". Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly. Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year. So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring". Add tests
for the new range format. Set the default cutoff for the start of Spring season will be the last Sunday in February
2 years ago
Gabe Farrell
b3e6efdb2e
Faster admin data loader + rename School.school_hash
2 years ago
c6c9fd38de
fix: batch in smaller size to stay within memory limits
2 years ago
e7f06ef4e9
chore: remove lowell seeding rules
2 years ago
063810a9d7
chore: make sure to load races in data loader instead of student loader
2 years ago
381625f58b
chore: reenable test spec that tests data loader for races
2 years ago
317fe3239a
chore: refactor Race out of survey_item_values
2 years ago
a295b8afb9
chore: refactor Gender out out of survey_item_values row
2 years ago
acfdaf5587
feat: add special education disaggregation
2 years ago
245cde85cd
Add disaggregation by ELL
2 years ago
76ebcc6ef3
feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page
2 years ago
22cc303a95
fix: There was an n+1 problem where we looked up the list of schools for
...
every row. Now we query the list of schools just once per file
2 years ago
784e23982e
feat: update survey_item_response table to indlude recorded date and import recorded date when loading responses
2 years ago
6b2bceceb6
Fix: Parse headers when they are surrounded by quotes. This helps load recent csv files correctly
3 years ago
37e932e078
Fix regression in student loader
3 years ago
0dfc9726d0
Add automated data cleaning. Modify SurveyItemValues class to use regex
...
instead of hard coded values. Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
3 years ago
a066f464c7
fix failing tests
3 years ago
a5da0fb0c6
Fix bug with not all survey responses loading when using sftp loader
3 years ago
47c1856281
Process 1000 rows at a time to limit memory usage in production
3 years ago
380048f5cd
Load survey responses from sftp folder
3 years ago
a1b7db9b2e
Add ability to load student demographic information for only lowell
3 years ago
fc9e73f656
make sure survey responses for Butler get loaded correctly
3 years ago
9bbba64e9e
Add rules to the survey data loader to allow skipping non-lowell schools
3 years ago
43bc55ed32
Query for survey items once instead of on each row
3 years ago
83fa255e5d
Make utility method private
3 years ago
3ca270e3b1
Fix more malformed csv errors
3 years ago
360579852c
Send fewer sql queries when loading survey responses
3 years ago
9ae94a2dae
further reduce batch size to avoid exceeding memory quota
3 years ago
bb472ce6c3
Reduce number of rows to process to avoid exceeding memory limit on heroku
3 years ago
7d7bee6498
Add gender disagreggation
3 years ago
a1f3dfaa33
Load gender information to survey item responses
3 years ago
f9c4ed8880
make sure grade gets updated when loading survey responses
3 years ago
89a7f27b88
Finishes #183092367 . Adds ui and graphs to show grade level disaggregations
3 years ago
bb5f668497
Refactor code based on rubocop and reek suggestions
3 years ago
18b479b8b2
Refactor based on rubocop and reek suggestions
3 years ago
2d945fb696
Print a message when a survey item response is rejected from database import due to an invalid likert score
4 years ago
6f629e2feb
Add survey responses from 2016 through 2018. Finishes #181255948
4 years ago
101c252896
Import survey item responses based on DESE ID instead of qualtrics school code. Finishes #181704978
4 years ago
36da912253
Modify survey item response csv files for 2019-20 to correct incorrect likert scores
4 years ago
ed7a3b8a3d
Autoformat code with rubocop
4 years ago
ad03606d66
Add benchmarks to survey and admin data items. Remove them from measures. Modify seeder
...
Calculate benchmarks for measures based on a weighted average of survey
and admin data items
Added architectural records
4 years ago
65ee27742a
Perform single lookup for survey items rather than n lookups
4 years ago
6477792f1d
Improve efficiency when checking whether survey item responses already exist
...
- see https://semaphoreci.com/blog/2017/03/14/faster-rails-how-to-check-if-a-record-exists.html for more info if curious
4 years ago
666eec9a20
Removed print statements from data loader output
4 years ago
b4f6daa406
Process CSV file rows in batches
4 years ago
a269bb0ec1
Try reporting on batch inserts
4 years ago
a41d5bb4c9
Update survey data loader for new student survey responses csv
4 years ago
Nelson Jovel
Gabe Farrell
Liam Morley
Alex Basson