Commit graph

33 commits

Author SHA1 Message Date
rebuilt
b6675c2516 ECP-206 Delete the 3 info boxes for each sub-category on the Browse page 2025-10-30 13:32:39 -07:00
rebuilt
5b00454a1b ECP-125 feat:
Update cleaner to read parent races.
Update uploader to set races for parents.
Add race graphs to analyze page.  Show both measure and scale level graphs
2025-05-22 11:57:26 -07:00
rebuilt
0f457becf0 feat: create a parents by language graph
Update demographics table with lanugage options

Create a lanugage table to hold the new languages

Update the demographic loader to input languages into the database

Update the cleaner to read the language column

Update the parent table to hold a reference to a language

Update the data uploader script to read the language from the csv and update the language information for any parent items that already exist (or create database entries if none already exist)

update the analyze interface to add controls for selecting ‘parents by group’ and a dropdown for ‘parent by language’

Update the analyze controller to read the parent-by-group parameter

Create a graph for the parent-by-group view

Bubble up averages for language calculations.

Make sure n-size only counts responses for a given measure.
2025-04-28 16:42:11 -07:00
Nelson Jovel
7e98f7446c fix: tests 2024-11-26 10:17:36 -08:00
Nelson Jovel
beb93aa8e8 fix: make sure all likert scores get counted even when the survey item id has different capitalization. Add tests for uploading parent data. Change the parent response rate calcuation to count all students in the school instead of just for the grades that were given the student survey 2024-11-08 15:36:34 -08:00
Nelson Jovel
3ecc68edd0 chore: correct parsing for 'not sped' and 'lep not first year' 2024-06-26 12:02:22 -07:00
Nelson Jovel
b1f942133b update parsing rules from glossary 2024-05-15 10:15:16 -07:00
Nelson Jovel
ea2feb138b add disaggregation glossary 2024-05-15 10:15:16 -07:00
Nelson Jovel
927fae1afd chore: add a test for categorizing sped values of 1 and 0 as 'Special Education' and 'Not Special Education' 2024-05-07 19:38:08 -07:00
Nelson Jovel
8bb6f5e8f0 Add ell income and sped parsing rules for SIS data. Add tests for the
new inputs.
2024-05-07 18:46:53 -07:00
Nelson Jovel
33da0859b9 Split academic year into seasons if the academic year's range is
initialized with a season, i.e. "2024-25 Fall".  Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly.   Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year.  So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring".  Add tests
for the new range format.  Set the default cutoff for the start of Spring season will be the last Sunday in February
2024-04-25 09:21:04 -07:00
5789ebf564 Faster admin data loader + rename School.school_hash 2024-04-22 14:46:37 -04:00
Nelson Jovel
289b04bc69 match an additional format for Dates. Supported dates are now '1/10/2022 14:21:45' '2022-1-10T14:21:45' '2022-1-10 14:21:45' 2024-03-01 09:30:23 -08:00
Nelson Jovel
d6735d449d feat: Support two date formats: ISO 8601 and the standard US date format
used in google sheets
2024-02-27 11:55:47 -08:00
Nelson Jovel
ed07114a91 fix: fix failing tests 2024-02-22 10:41:42 -08:00
Nelson Jovel
36e21515c3 chore: refactor Race out of survey_item_values 2023-12-20 12:25:23 -08:00
Nelson Jovel
e7fb009425 chore: refactor Gender out out of survey_item_values row 2023-12-20 11:08:23 -08:00
Nelson Jovel
41d942c214 chore: Make sure 'hispanic' column only gets applied when using SIS race information 2023-12-12 10:53:07 -08:00
Nelson Jovel
d90a83e510 fix: instead of looking for 'asian' at the start of a word, look for it
after a word boundary.  This means it still doesn't get confused with
caucasian and it's more flexible whan asian appears inside other text
such as 'Caucasian and Asian and Black'
2023-12-08 14:16:50 -08:00
Nelson Jovel
3f44613085 chore: various fixes for race and gender categorization during cleaning.
Also add tests for race and gender categorization
2023-12-08 13:12:19 -08:00
Nelson Jovel
b7e670bb60 Lower threshold for the number of valid student responses from 17 to 11 2023-12-06 14:15:19 -08:00
Nelson Jovel
e325f38c43 Convert gender and race text into qualtrics codes during cleaning. Abide by 'prefer not to disclose' for self reported race. Give priority to self reported data but use SIS information as backup 2023-12-06 14:10:16 -08:00
Nelson Jovel
b63c327d33 chore: when searching for dese id, split up pattern so that to be more explicit about the order in which to search out the columns that might have the dese ID we're looking for. 2023-11-06 13:15:50 -08:00
rebuilt
e3fbbabce5 feat: We no longer trust the progress number that gets exported from qualtrics. Instead during the cleaning progress, perform a manual count of the number of responses to filter out rows that don't meet the minimum threshold. 2023-10-27 15:12:24 -07:00
rebuilt
48e795fcfb feat: add special education disaggregation 2023-10-06 11:41:52 -07:00
rebuilt
060d7aa55a Add disaggregation by ELL 2023-09-29 19:29:23 -07:00
rebuilt
a785c69c44 Add Overall Response Rate 2023-08-09 15:13:58 -07:00
rebuilt
4f035f6a63 feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page 2023-07-07 09:14:36 -07:00
Nelson Jovel
0a2c5e02c5 feat: add ability to merge disaggregation data with raw survey data to
produce a cleaned csv with merged income disaggregation columns
2023-06-20 12:22:24 -07:00
rebuilt
9aeb5f92af Missing progress or duration information does not result in a row removed in the cleaning process 2023-06-02 15:23:21 -07:00
rebuilt
e3ae12b425 update response_date to recorded_date 2023-05-31 16:57:47 -07:00
rebuilt
8ef8cfce58 Adjust valid duration threshold of short form items 2023-05-26 18:30:44 -07:00
rebuilt
4509c157fa Add automated data cleaning. Modify SurveyItemValues class to use regex
instead of hard coded values.  Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-16 13:38:29 -07:00