Commit graph

21 commits

Author SHA1 Message Date
rebuilt
f1022728fa Fix problem with dese scraper lumping in 2021-22 data as 2022-23 data.
Deleted unused csvs.  Turned off puts statements in admin loader.
Remove old, now unused admin data loader class.
2023-04-30 12:04:20 -07:00
rebuilt
cee7aa4c59 Remove unused filename 2023-04-22 14:01:21 -07:00
rebuilt
d3a28f7635 Fix ThreeATwo scraper 2023-04-22 14:01:01 -07:00
rebuilt
07ed8dd259 Update logic for calculating student response rate. Remove references
to survey table.  We no longer check or keep track of the survey type.
Instead we look in the database to see if a survey item has at least 10
responses.  If it does, that survey item was presented to the respondent
and we count it, and all responses when calculating the response rate.

Remove response rate timestamp from caching logic because we no longer
add the response rate to the database. All response rates are calculated
on the fly

Update three_b_two scraper to use teacher only numbers

swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information
2023-04-22 14:00:20 -07:00
rebuilt
ed421296c7 Fix bug that caused an incorrect score of 1 to be assigned for admin
data item related to parity index of teachers of color to students of
color.  Now if there's not enough data to calculate a score, it produces
'NA' instead of 1. Also added admin data for a-reso-i1 and a-curv-i1
2023-01-06 17:43:48 -08:00
rebuilt
7b9abe3793 Scrape latest data from DESE 2022-12-20 15:26:11 -08:00
rebuilt
f4e2e6ca83 Correct headers for 4D so the data can be correctly imported 2022-10-21 17:29:19 -07:00
rebuilt
dd9ed86dcd Merge branch 'scrapers' 2022-09-21 19:54:24 -07:00
rebuilt
f556844b73 Update a-sust-i1 so it's only applicable to high schools and lower
benchmark for a-cure-i1 from 6.7 to 5 percent.
2022-09-21 19:50:33 -07:00
rebuilt
9e82023e48 Make sure we don't divide by 0 2022-09-20 16:44:11 -07:00
rebuilt
1a984d2cf0 Make sure we don't divide by 0 2022-09-20 16:43:19 -07:00
rebuilt
3ee909bbea Update 3B-2 with an absolute floor 2022-09-20 14:45:20 -07:00
rebuilt
1be6e79fd8 Add scrapers for admin data and their corresponding csv files of data.
Make sure specs are off so they don't run in github CI
2022-09-19 19:37:39 -07:00
rebuilt
9e18bf2151 Turn off specs for scrapers. Generate csvs for scraped data 2022-09-13 22:59:04 -07:00
rebuilt
75cea539c5 Change scraped csv file names from word to number format 'one_a_one.csv'
-> '1A_1.csv''
2022-09-13 10:46:18 -07:00
rebuilt
3e28be2d0d Remove 'scraper' from file/class names add 3B-1 2022-09-12 11:24:57 -07:00
rebuilt
83ef9310a4 Add scraper for 3A-1 and 2C-1. Finishes #183105787 and #183105716 2022-09-07 11:34:42 -07:00
rebuilt
b5bc9c9de4 Add 2A-i admin data. Finishes #183105627 2022-09-05 22:37:53 -07:00
rebuilt
459393381b Load admin data for 1A-iii. Finishes #183105516 2022-09-05 21:01:32 -07:00
rebuilt
f783ce9724 add one-a 2022-09-01 20:00:20 -07:00
rebuilt
048aacd641 Add admin data for 4D. Finishes #182818489 and #182793402 2022-08-30 21:46:23 -07:00