4f035f6a63
feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page
2 years ago
Gabe Farrell
527aa66796
Patch upload script over from rpp
3 years ago
Gabe Farrell
b07a675427
Add cleaned data upload script for Lowell
3 years ago
0a2c5e02c5
feat: add ability to merge disaggregation data with raw survey data to
...
produce a cleaned csv with merged income disaggregation columns
3 years ago
30285efd69
It's possible for admin data likert score values to be above 5. If that happens, we
...
cap the likert score at 5. This was happening already at the scraper
level but it's also now being done by the admin data loader for safety.
Also make sure to just update admin data instead of deleting and
reloading all values each load. Add tests to confirm this behavior
3 years ago
87ca23df79
Add subcategory report
3 years ago
4509c157fa
Add automated data cleaning. Modify SurveyItemValues class to use regex
...
instead of hard coded values. Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
3 years ago
5cf5a5f383
added tasks to find stray responses and skipped questions
3 years ago
fb963f63e0
Clean up rake tasks to get data:load_survey_responses working
3 years ago
128748addd
Update logic for calculating student response rate. Remove references
...
to survey table. We no longer check or keep track of the survey type.
Instead we look in the database to see if a survey item has at least 10
responses. If it does, that survey item was presented to the respondent
and we count it, and all responses when calculating the response rate.
Remove response rate timestamp from caching logic because we no longer
add the response rate to the database. All response rates are calculated
on the fly
Update three_b_two scraper to use teacher only numbers
swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information
3 years ago
a93b79151c
update response load task for maynard
3 years ago
a656441fcd
Fix oneoff script for ECP so it doesn't skip non-lowell schools
3 years ago
d9ed53c846
also set response rate thresholds
3 years ago
7aac39340b
and one off script to temporarily set student and teacher response rates for lee to 100 percent
3 years ago
35cbb640ec
add one-off script to load lee data
3 years ago
4c4ccc01cc
Merge branch 'rpp-response-rate' to bring in changes to test files
3 years ago
06f9d2f0e9
Scrape enrollment and staffing information. Seed enrollment and staffing information. Update DatabaseCleaner so it cleans up leftover information in the database. Remove old admin csvs from codebase.
3 years ago
c0332955f3
move csv require statement to application.rb
3 years ago
ae6d7151b3
Load student respondent information from enrollment loader
3 years ago
984fedff02
Add academic year column to enrollment file. Add enrollment loader to seeder file
3 years ago
83ab2cfef0
Add one_off script for loading 2022-23 data
3 years ago
669fb7f1f0
Correct grade level for kindergarten and add one off task to load test data
3 years ago
3a744db975
Start adding kindergarten
3 years ago
03c6bff7bb
Load survey responses from sftp folder
3 years ago
eeeb4f5d14
Add Stoklosa survey responses
3 years ago
d97b62b527
update logo and hero image
3 years ago
e2bf24e883
Add rules to the survey data loader to allow skipping non-lowell schools
3 years ago
a2caf42fee
merge disagreggation features
3 years ago
e8825c6796
fix rake task for uploading responses so that it also loads student racial information
3 years ago
8c8dc39465
Also delete response rates for non lowell schools
3 years ago
7b9abe3793
Scrape latest data from DESE
3 years ago
0500d7c93f
Add a way to delete non-lowell schools and districts
3 years ago
a1b7db9b2e
Add ability to load student demographic information for only lowell
3 years ago
d49cf918ac
Add a way to seed only lowell schools
3 years ago
9bbba64e9e
Add rules to the survey data loader to allow skipping non-lowell schools
3 years ago
0321b3d43f
add tasks to upload single year of data
3 years ago
89a7f27b88
Finishes #183092367 . Adds ui and graphs to show grade level disaggregations
3 years ago
3e28be2d0d
Remove 'scraper' from file/class names add 3B-1
3 years ago
048aacd641
Add admin data for 4D. Finishes #182818489 and #182793402
3 years ago
30eae881f3
Load correct column for 2021-22 year
3 years ago
bc63dfab7e
add one_off scripts for each academic year
3 years ago
3fcab58557
Do not erase all students when loading each file
3 years ago
3f493727b3
Load a single year of student demographic data and race scores at a time
3 years ago
d89a207c31
Load student data for all years and calculate race scores for all
3 years ago
fc4fbb4644
load students from all years
3 years ago
105f30f220
Change race score calculations so the average bubbles up through survey_items
...
-> scales -> measure.
Precalculate averages for performance.
3 years ago
50b3bc678c
Columns for students by group graph change when corresponding race checkboxes are clicked
3 years ago
93486eb132
Fix failing tests
3 years ago
94a58a87d7
Use modules to namespace classes for analyze page
3 years ago
d44a5275b4
Address bug related to story #182855028
3 years ago
rebuilt
Gabe Farrell
vineeshathallapaneni@gmail.com