Commit graph

186 commits

Author SHA1 Message Date
rebuilt
1e3cebf223 add oneoff script to load wareham data 2023-05-27 17:05:40 -07:00
rebuilt
28e7f1fcb2 Change Dese ID of Minot Forest in Wareham to 310001799. Change Dese ID of Wareham Elementary School to 3100017 2023-05-27 15:44:04 -07:00
rebuilt
0dfc9726d0 Add automated data cleaning. Modify SurveyItemValues class to use regex
instead of hard coded values.  Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-24 11:59:53 -07:00
rebuilt
b5dc933187 Add subcategory report 2023-05-23 18:43:55 -07:00
rebuilt
03efd88284 Do not create response rate entries in the database when loading survey item responses 2023-04-25 17:16:44 -07:00
rebuilt
07ed8dd259 Update logic for calculating student response rate. Remove references
to survey table.  We no longer check or keep track of the survey type.
Instead we look in the database to see if a survey item has at least 10
responses.  If it does, that survey item was presented to the respondent
and we count it, and all responses when calculating the response rate.

Remove response rate timestamp from caching logic because we no longer
add the response rate to the database. All response rates are calculated
on the fly

Update three_b_two scraper to use teacher only numbers

swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information
2023-04-22 14:00:20 -07:00
rebuilt
098110433a Remove duplicate line 2023-03-23 19:57:36 -07:00
rebuilt
5f58a5516e print the number of student and teacher respondents 2023-03-23 17:29:02 -07:00
rebuilt
36ea63332b Add 2022-23 survey results 2023-03-17 18:55:35 -07:00
rebuilt
549d0eaa22 Add script to load 2016-18 data 2023-02-19 19:36:14 -08:00
rebuilt
6d770938ef Swap survey data loading from local files to SFTP 2023-02-19 19:36:14 -08:00
rebuilt
14570b3c33 update tasks so that data gets loaded from sftp. Add task to delete 2016-17, and 2017-18 data 2023-02-19 19:36:14 -08:00
rebuilt
380048f5cd Load survey responses from sftp folder 2023-02-19 19:36:14 -08:00
rebuilt
5118358b43 Also run student demographic loader when loading survey data 2023-01-16 17:04:12 -08:00
rebuilt
e8825c6796 fix rake task for uploading responses so that it also loads student racial information 2023-01-06 17:43:30 -08:00
rebuilt
8c8dc39465 Also delete response rates for non lowell schools 2022-12-21 11:22:54 -08:00
rebuilt
7b9abe3793 Scrape latest data from DESE 2022-12-20 15:26:11 -08:00
rebuilt
0500d7c93f Add a way to delete non-lowell schools and districts 2022-12-14 14:18:54 -08:00
rebuilt
a1b7db9b2e Add ability to load student demographic information for only lowell 2022-12-03 11:33:11 -08:00
rebuilt
d49cf918ac Add a way to seed only lowell schools 2022-12-03 10:23:02 -08:00
rebuilt
9bbba64e9e Add rules to the survey data loader to allow skipping non-lowell schools 2022-12-01 16:15:45 -08:00
rebuilt
0321b3d43f add tasks to upload single year of data 2022-11-23 11:01:44 -08:00
rebuilt
89a7f27b88 Finishes #183092367. Adds ui and graphs to show grade level disaggregations 2022-10-16 22:03:25 -07:00
rebuilt
3e28be2d0d Remove 'scraper' from file/class names add 3B-1 2022-09-12 11:24:57 -07:00
rebuilt
048aacd641 Add admin data for 4D. Finishes #182818489 and #182793402 2022-08-30 21:46:23 -07:00
rebuilt
30eae881f3 Load correct column for 2021-22 year 2022-08-22 22:09:07 -07:00
rebuilt
bc63dfab7e add one_off scripts for each academic year 2022-08-22 15:24:15 -07:00
rebuilt
3fcab58557 Do not erase all students when loading each file 2022-08-22 14:36:00 -07:00
rebuilt
3f493727b3 Load a single year of student demographic data and race scores at a time 2022-08-22 11:29:38 -07:00
rebuilt
d89a207c31 Load student data for all years and calculate race scores for all 2022-08-20 15:07:15 -07:00
rebuilt
fc4fbb4644 load students from all years 2022-08-16 18:25:18 -07:00
rebuilt
105f30f220 Change race score calculations so the average bubbles up through survey_items
-> scales -> measure.
Precalculate averages for performance.
2022-08-10 09:58:11 -07:00
rebuilt
50b3bc678c Columns for students by group graph change when corresponding race checkboxes are clicked 2022-08-04 21:21:22 -07:00
rebuilt
93486eb132 Fix failing tests 2022-08-03 11:11:26 -07:00
rebuilt
94a58a87d7 Use modules to namespace classes for analyze page 2022-08-02 18:16:33 -07:00
rebuilt
d44a5275b4 Address bug related to story #182855028 2022-07-29 16:04:43 -07:00
rebuilt
12e4e3f177 import students 2022-07-29 15:34:56 -07:00
rebuilt
18b479b8b2 Refactor based on rubocop and reek suggestions 2022-07-08 19:07:23 -07:00
rebuilt
dfad41bdd5 Load teacher responses for lowell and milford 2021-22 2022-06-22 21:06:23 -07:00
rebuilt
3610d5f22d add 2021-22 data for lowell and milford 2022-06-22 18:40:10 -07:00
rebuilt
d0dcac701e change wording of output message in one-off script 2022-06-17 14:27:30 -07:00
rebuilt
7fcf580591 Load survey responses for revere somerville and wareham for the 2021-22 academic year 2022-06-17 13:45:10 -07:00
rebuilt
2d3ee28cc3 Change wording of refreshing response rates to resetting response rates 2022-06-16 17:22:26 -07:00
rebuilt
a6ad132c81 Add counter caches 2022-06-16 16:14:03 -07:00
rebuilt
c03615cb43 Rename ResponseRate to ResponseRateCalculator. Create a new response
rate model.  Create a loader to refresh response rates for all
subcategories.

Use precalculated response rates in views

Wrap more elements in page caching

Calculate a response rate for a subcategory if one does not already
exist
2022-06-16 08:19:26 -07:00
rebuilt
c63e1ebd8e Add 2021-22 winchester student and teacher surveys. Finishes #182376738 2022-06-09 11:16:05 -07:00
Nelson Jovel
d7b0fe0e36 Add attleboro survey item responses for 2021-22. Finishes #182215755 2022-05-18 13:14:01 -07:00
Nelson Jovel
9768d68385 Add task to reset admin data values 2022-05-13 13:09:54 -07:00
Nelson Jovel
e27238e3c4 Correct student survey responses from 2016-17. Finishes #182076798 2022-05-10 14:58:53 -07:00
Nelson Jovel
bc27e5a0e8 Add DESE_ID column to 2018-19 boston mattahunt teacher survey csv so survey item responses will load correctly 2022-05-03 10:54:29 -07:00