Commit graph

200 commits

Author SHA1 Message Date
Nelson Jovel
a70a6ae32c chore: update source of truth and add a way to delete 'Library Access' measure from database 2024-03-08 14:42:53 -08:00
Nelson Jovel
8dfaa86982 feat: create a score csv report for measures 2024-03-06 15:50:07 -08:00
Nelson Jovel
76bd79e8a2 chore: clean up rake tasks 2023-12-20 13:47:38 -08:00
rebuilt
79bc91d39f feat: add command for loading survey responses from arbitrary sftp directory 2023-10-30 19:44:52 -07:00
rebuilt
245cde85cd Add disaggregation by ELL 2023-10-17 11:18:47 -07:00
rebuilt
071e7b5208 chore: continue cleaning up unneeded tasks 2023-08-09 11:01:46 -07:00
rebuilt
2903b455ce chore: remove unused tasks from data.rake and one_off.rake 2023-08-08 16:39:29 -07:00
rebuilt
76ebcc6ef3 feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page 2023-08-08 15:52:12 -07:00
vineeshathallapaneni@gmail.com
1fc1c8348f Revert "added a rake task to output stray responses"
This reverts commit ac0374533e.
2023-06-28 23:55:35 -04:00
vineeshathallapaneni@gmail.com
ac0374533e added a rake task to output stray responses 2023-06-28 23:51:00 -04:00
73a5f4e30b Patch upload script over from rpp 2023-06-21 03:27:47 +00:00
4091fcdd44 Add cleaned data upload script for Lowell 2023-06-21 03:23:00 +00:00
rebuilt
904d0d2f2c It's possible for admin data likert score values to be above 5. If that happens, we
cap the likert score at 5.   This was happening already at the scraper
level but it's also now being done by the admin data loader for safety.
Also make sure to just update admin data instead of deleting and
reloading all values each load. Add tests to confirm this behavior
2023-06-03 16:06:50 -07:00
rebuilt
83fa0c4e63 Removed duplicate parameter 2023-06-02 10:36:46 -07:00
rebuilt
1e3cebf223 add oneoff script to load wareham data 2023-05-27 17:05:40 -07:00
rebuilt
28e7f1fcb2 Change Dese ID of Minot Forest in Wareham to 310001799. Change Dese ID of Wareham Elementary School to 3100017 2023-05-27 15:44:04 -07:00
rebuilt
0dfc9726d0 Add automated data cleaning. Modify SurveyItemValues class to use regex
instead of hard coded values.  Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-24 11:59:53 -07:00
rebuilt
b5dc933187 Add subcategory report 2023-05-23 18:43:55 -07:00
rebuilt
03efd88284 Do not create response rate entries in the database when loading survey item responses 2023-04-25 17:16:44 -07:00
rebuilt
07ed8dd259 Update logic for calculating student response rate. Remove references
to survey table.  We no longer check or keep track of the survey type.
Instead we look in the database to see if a survey item has at least 10
responses.  If it does, that survey item was presented to the respondent
and we count it, and all responses when calculating the response rate.

Remove response rate timestamp from caching logic because we no longer
add the response rate to the database. All response rates are calculated
on the fly

Update three_b_two scraper to use teacher only numbers

swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information
2023-04-22 14:00:20 -07:00
rebuilt
098110433a Remove duplicate line 2023-03-23 19:57:36 -07:00
rebuilt
5f58a5516e print the number of student and teacher respondents 2023-03-23 17:29:02 -07:00
rebuilt
36ea63332b Add 2022-23 survey results 2023-03-17 18:55:35 -07:00
rebuilt
549d0eaa22 Add script to load 2016-18 data 2023-02-19 19:36:14 -08:00
rebuilt
6d770938ef Swap survey data loading from local files to SFTP 2023-02-19 19:36:14 -08:00
rebuilt
14570b3c33 update tasks so that data gets loaded from sftp. Add task to delete 2016-17, and 2017-18 data 2023-02-19 19:36:14 -08:00
rebuilt
380048f5cd Load survey responses from sftp folder 2023-02-19 19:36:14 -08:00
rebuilt
5118358b43 Also run student demographic loader when loading survey data 2023-01-16 17:04:12 -08:00
rebuilt
e8825c6796 fix rake task for uploading responses so that it also loads student racial information 2023-01-06 17:43:30 -08:00
rebuilt
8c8dc39465 Also delete response rates for non lowell schools 2022-12-21 11:22:54 -08:00
rebuilt
7b9abe3793 Scrape latest data from DESE 2022-12-20 15:26:11 -08:00
rebuilt
0500d7c93f Add a way to delete non-lowell schools and districts 2022-12-14 14:18:54 -08:00
rebuilt
a1b7db9b2e Add ability to load student demographic information for only lowell 2022-12-03 11:33:11 -08:00
rebuilt
d49cf918ac Add a way to seed only lowell schools 2022-12-03 10:23:02 -08:00
rebuilt
9bbba64e9e Add rules to the survey data loader to allow skipping non-lowell schools 2022-12-01 16:15:45 -08:00
rebuilt
0321b3d43f add tasks to upload single year of data 2022-11-23 11:01:44 -08:00
rebuilt
89a7f27b88 Finishes #183092367. Adds ui and graphs to show grade level disaggregations 2022-10-16 22:03:25 -07:00
rebuilt
3e28be2d0d Remove 'scraper' from file/class names add 3B-1 2022-09-12 11:24:57 -07:00
rebuilt
048aacd641 Add admin data for 4D. Finishes #182818489 and #182793402 2022-08-30 21:46:23 -07:00
rebuilt
30eae881f3 Load correct column for 2021-22 year 2022-08-22 22:09:07 -07:00
rebuilt
bc63dfab7e add one_off scripts for each academic year 2022-08-22 15:24:15 -07:00
rebuilt
3fcab58557 Do not erase all students when loading each file 2022-08-22 14:36:00 -07:00
rebuilt
3f493727b3 Load a single year of student demographic data and race scores at a time 2022-08-22 11:29:38 -07:00
rebuilt
d89a207c31 Load student data for all years and calculate race scores for all 2022-08-20 15:07:15 -07:00
rebuilt
fc4fbb4644 load students from all years 2022-08-16 18:25:18 -07:00
rebuilt
105f30f220 Change race score calculations so the average bubbles up through survey_items
-> scales -> measure.
Precalculate averages for performance.
2022-08-10 09:58:11 -07:00
rebuilt
50b3bc678c Columns for students by group graph change when corresponding race checkboxes are clicked 2022-08-04 21:21:22 -07:00
rebuilt
93486eb132 Fix failing tests 2022-08-03 11:11:26 -07:00
rebuilt
94a58a87d7 Use modules to namespace classes for analyze page 2022-08-02 18:16:33 -07:00
rebuilt
d44a5275b4 Address bug related to story #182855028 2022-07-29 16:04:43 -07:00