071e7b5208
chore: continue cleaning up unneeded tasks
2 years ago
2903b455ce
chore: remove unused tasks from data.rake and one_off.rake
2 years ago
76ebcc6ef3
feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page
2 years ago
1fc1c8348f
Revert "added a rake task to output stray responses"
...
This reverts commit ac0374533e
2 years ago
ac0374533e
added a rake task to output stray responses
2 years ago
Gabe Farrell
73a5f4e30b
Patch upload script over from rpp
3 years ago
Gabe Farrell
4091fcdd44
Add cleaned data upload script for Lowell
3 years ago
904d0d2f2c
It's possible for admin data likert score values to be above 5. If that happens, we
...
cap the likert score at 5. This was happening already at the scraper
level but it's also now being done by the admin data loader for safety.
Also make sure to just update admin data instead of deleting and
reloading all values each load. Add tests to confirm this behavior
3 years ago
83fa0c4e63
Removed duplicate parameter
3 years ago
1e3cebf223
add oneoff script to load wareham data
3 years ago
28e7f1fcb2
Change Dese ID of Minot Forest in Wareham to 310001799. Change Dese ID of Wareham Elementary School to 3100017
3 years ago
0dfc9726d0
Add automated data cleaning. Modify SurveyItemValues class to use regex
...
instead of hard coded values. Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
3 years ago
b5dc933187
Add subcategory report
3 years ago
03efd88284
Do not create response rate entries in the database when loading survey item responses
3 years ago
07ed8dd259
Update logic for calculating student response rate. Remove references
...
to survey table. We no longer check or keep track of the survey type.
Instead we look in the database to see if a survey item has at least 10
responses. If it does, that survey item was presented to the respondent
and we count it, and all responses when calculating the response rate.
Remove response rate timestamp from caching logic because we no longer
add the response rate to the database. All response rates are calculated
on the fly
Update three_b_two scraper to use teacher only numbers
swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information
3 years ago
098110433a
Remove duplicate line
3 years ago
5f58a5516e
print the number of student and teacher respondents
3 years ago
36ea63332b
Add 2022-23 survey results
3 years ago
549d0eaa22
Add script to load 2016-18 data
3 years ago
6d770938ef
Swap survey data loading from local files to SFTP
3 years ago
14570b3c33
update tasks so that data gets loaded from sftp. Add task to delete 2016-17, and 2017-18 data
3 years ago
380048f5cd
Load survey responses from sftp folder
3 years ago
5118358b43
Also run student demographic loader when loading survey data
3 years ago
e8825c6796
fix rake task for uploading responses so that it also loads student racial information
3 years ago
8c8dc39465
Also delete response rates for non lowell schools
3 years ago
7b9abe3793
Scrape latest data from DESE
3 years ago
0500d7c93f
Add a way to delete non-lowell schools and districts
3 years ago
a1b7db9b2e
Add ability to load student demographic information for only lowell
3 years ago
d49cf918ac
Add a way to seed only lowell schools
3 years ago
9bbba64e9e
Add rules to the survey data loader to allow skipping non-lowell schools
3 years ago
0321b3d43f
add tasks to upload single year of data
3 years ago
89a7f27b88
Finishes #183092367 . Adds ui and graphs to show grade level disaggregations
3 years ago
3e28be2d0d
Remove 'scraper' from file/class names add 3B-1
3 years ago
048aacd641
Add admin data for 4D. Finishes #182818489 and #182793402
3 years ago
30eae881f3
Load correct column for 2021-22 year
3 years ago
bc63dfab7e
add one_off scripts for each academic year
3 years ago
3fcab58557
Do not erase all students when loading each file
3 years ago
3f493727b3
Load a single year of student demographic data and race scores at a time
3 years ago
d89a207c31
Load student data for all years and calculate race scores for all
3 years ago
fc4fbb4644
load students from all years
3 years ago
105f30f220
Change race score calculations so the average bubbles up through survey_items
...
-> scales -> measure.
Precalculate averages for performance.
3 years ago
50b3bc678c
Columns for students by group graph change when corresponding race checkboxes are clicked
3 years ago
93486eb132
Fix failing tests
3 years ago
94a58a87d7
Use modules to namespace classes for analyze page
3 years ago
d44a5275b4
Address bug related to story #182855028
3 years ago
12e4e3f177
import students
3 years ago
18b479b8b2
Refactor based on rubocop and reek suggestions
3 years ago
dfad41bdd5
Load teacher responses for lowell and milford 2021-22
4 years ago
3610d5f22d
add 2021-22 data for lowell and milford
4 years ago
d0dcac701e
change wording of output message in one-off script
4 years ago
rebuilt
vineeshathallapaneni@gmail.com
Gabe Farrell