Nelson Jovel
45ea735fa5
perf: reduce number of queries for teacher survey_items
2024-01-12 13:53:41 -08:00
Nelson Jovel
3718d74e62
perf: reduce number of queries for respondents
2024-01-12 13:53:33 -08:00
Nelson Jovel
277e36429a
perf: reduce number of queries by requesting grouped_responses once instead of for each measure
2024-01-12 13:52:41 -08:00
Nelson Jovel
5b34c2257f
chore: performance fixes for response rate calculation
2024-01-12 13:51:59 -08:00
Nelson Jovel
47cca3ce1f
fix: reduce number of n+1 queries
2024-01-12 13:51:14 -08:00
Nelson Jovel
2dd73f003c
chore: remove method that wasn't needed
2024-01-12 13:49:11 -08:00
Nelson Jovel
a960202d66
chore: performance fixes for response rate calculation
2024-01-12 13:49:02 -08:00
Nelson Jovel
9e368ae47d
chore: prevent multiple lookups for the boston district by inlining the
...
sql
2024-01-12 13:48:51 -08:00
Nelson Jovel
db7c74479d
chore: memoize grades for school
2024-01-12 13:48:08 -08:00
Nelson Jovel
d2efec1574
fix: reduce number of n+1 queries
2023-12-21 13:35:48 -08:00
Nelson Jovel
2a125aa058
chore: bring in some refactoring changes that didn't get copied over from the main branch
2023-12-20 19:27:06 -08:00
Nelson Jovel
75451648e5
chore: move code that normalizes race data to the Race class
2023-12-20 19:07:52 -08:00
Nelson Jovel
0359dae88a
chore: rename counts_by_grade to enrollment_by_grade for clarity
2023-12-18 15:59:30 -08:00
Nelson Jovel
d79635054c
fix: instead of looking for 'asian' at the start of a word, look for it
...
after a word boundary. This means it still doesn't get confused with
caucasian and it's more flexible whan asian appears inside other text
such as 'Caucasian and Asian and Black'
2023-12-18 15:44:51 -08:00
Nelson Jovel
fe039e3d04
chore: various fixes for race and gender categorization during cleaning.
...
Also add tests for race and gender categorization
2023-12-18 15:38:31 -08:00
Nelson Jovel
054e5603b7
chore: fix failing legacy tests
2023-11-03 10:18:46 -07:00
rebuilt
9d680c5159
chore: upgrade to rails 7.1.
...
upgrade rspec
fix failing tests
upgrade devise
2023-10-17 12:34:11 -07:00
rebuilt
acfdaf5587
feat: add special education disaggregation
2023-10-17 12:29:00 -07:00
rebuilt
245cde85cd
Add disaggregation by ELL
2023-10-17 11:18:47 -07:00
rebuilt
2a8148b8cd
fix: hide scores on analyze page for scores that don't meet the student threshold of 25%
2023-08-22 12:18:18 -07:00
rebuilt
6db93cb116
feat: Rename income labels to 'Economically Disadvantaged' and 'Not Economically Disadvantaged'
2023-08-09 12:13:04 -07:00
rebuilt
a1b580048b
chore: remove precalculated race scores. Calculate race scores on every reload
2023-08-08 15:54:15 -07:00
rebuilt
76ebcc6ef3
feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page
2023-08-08 15:52:12 -07:00
rebuilt
7373e2e52f
fix: Filter out responses that don't correspond to the grades the school serves
2023-08-08 11:46:14 -07:00
rebuilt
0c18aa2083
chore: Add cypress to codebase
2023-07-19 12:43:56 -07:00
rebuilt
0ed3af8a1f
Add date range to subcategory report export
2023-06-23 11:50:12 -07:00
459e4b8d21
Use weighted average instead of simple average when calculating response rates
2023-06-21 04:45:53 +00:00
rebuilt
cee1c5fbc9
perf: Reduce number of queries. Remove unneeded methods. Combine
...
subcategory and measure queries in controller
2023-06-19 20:49:57 -07:00
rebuilt
f9d4b28969
perf: memoize function when calculating response rates
2023-06-10 11:17:40 -07:00
rebuilt
77e8d12a3b
perf: Add another index to surveyitemresponses. Implement bullet suggestions. Memoize a function in measure.rb
2023-06-10 10:19:05 -07:00
rebuilt
fb6f43e815
Modify subcategory report to be threaded
2023-06-09 21:12:04 -07:00
rebuilt
4dae0cb565
Add District, School Code, and Grade columns to subcategory report
2023-06-09 15:36:13 -07:00
rebuilt
81f8738d59
Fix: Ensure proper detection of survey type. Make sure to remove any survey item ids that end with the survey variant style of '-1'
2023-06-07 12:40:23 -07:00
rebuilt
6022739f07
use district short name when writing filename
2023-05-31 17:12:40 -07:00
rebuilt
0dfc9726d0
Add automated data cleaning. Modify SurveyItemValues class to use regex
...
instead of hard coded values. Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-24 11:59:53 -07:00
rebuilt
b5dc933187
Add subcategory report
2023-05-23 18:43:55 -07:00
rebuilt
c57ec6157c
Round all averages to two decimal places
2023-05-12 13:12:42 -07:00
rebuilt
6b2555c1f0
Fix response rates for small schools by accounting for expected minimum counts of responses less than 10
2023-05-12 12:27:46 -07:00
rebuilt
a066f464c7
fix failing tests
2023-04-30 16:35:08 -07:00
rebuilt
c42b569e78
cap response rate per grade at 100
2023-04-30 11:59:22 -07:00
rebuilt
0ed6cbe6b4
Rename DataAvailability to Summary
2023-04-22 14:24:39 -07:00
rebuilt
09963c3323
Remove surveys table from codebase since it's no longer needed after changing logic surrounding response rates and score calculations
2023-04-22 14:21:48 -07:00
rebuilt
30e006ee0d
Set a minimum threshold of 10 survey item responses to calculate scores per survey item
2023-04-22 14:00:51 -07:00
rebuilt
982f216c6c
rename counts_by_grade method to the more descriptive enrollment_by_grade
2023-04-22 14:00:40 -07:00
rebuilt
07ed8dd259
Update logic for calculating student response rate. Remove references
...
to survey table. We no longer check or keep track of the survey type.
Instead we look in the database to see if a survey item has at least 10
responses. If it does, that survey item was presented to the respondent
and we count it, and all responses when calculating the response rate.
Remove response rate timestamp from caching logic because we no longer
add the response rate to the database. All response rates are calculated
on the fly
Update three_b_two scraper to use teacher only numbers
swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information
2023-04-22 14:00:20 -07:00
rebuilt
049ae4548f
Don't add another response rate record to the database. Just calculate on the fly. Otherwise we can run into a situation where we create two response rates for a given school/academic_year/subcategory
2023-04-08 14:49:21 -07:00
rebuilt
380048f5cd
Load survey responses from sftp folder
2023-02-19 19:36:14 -08:00
rebuilt
72c90b8f47
Add selector for all data
2022-11-18 08:20:26 -08:00
rebuilt
7d7bee6498
Add gender disagreggation
2022-10-27 15:56:50 -07:00
rebuilt
a1f3dfaa33
Load gender information to survey item responses
2022-10-21 21:38:33 -07:00
