Commit graph

26 commits

Author SHA1 Message Date
rebuilt
9d8543afae feat: Update cleaner and uploader to load gender data from parent surveys
Implement graph views for parent gender data
2025-05-27 11:13:00 -07:00
rebuilt
5b00454a1b ECP-125 feat:
Update cleaner to read parent races.
Update uploader to set races for parents.
Add race graphs to analyze page.  Show both measure and scale level graphs
2025-05-22 11:57:26 -07:00
rebuilt
0f457becf0 feat: create a parents by language graph
Update demographics table with lanugage options

Create a lanugage table to hold the new languages

Update the demographic loader to input languages into the database

Update the cleaner to read the language column

Update the parent table to hold a reference to a language

Update the data uploader script to read the language from the csv and update the language information for any parent items that already exist (or create database entries if none already exist)

update the analyze interface to add controls for selecting ‘parents by group’ and a dropdown for ‘parent by language’

Update the analyze controller to read the parent-by-group parameter

Create a graph for the parent-by-group view

Bubble up averages for language calculations.

Make sure n-size only counts responses for a given measure.
2025-04-28 16:42:11 -07:00
rebuilt
3a5a368a35 feat: Update demographics file with housing statuses. Create housing class. Update survey_item_values.rb to parse housing info. Update cleaner to output housing info. 2025-03-27 15:20:40 -07:00
rebuilt
2fce07e3cd chore: correctly detect missing grade header 2025-03-19 10:52:35 -07:00
Nelson Jovel
e5e969b968 feat: add parent survey gauges 2024-09-16 15:24:43 -07:00
Nelson Jovel
ed3ac25a7b chore: During cleaning, stop execution if grade column isn't found. Also stop execution if a duplicate header is found. Turn off spec for duplicate header check 2024-05-23 12:52:13 -07:00
Nelson Jovel
c2c8ce3b8d chore: add a way to print out errors in dese id when cleaning files 2024-05-16 15:09:14 -07:00
Nelson Jovel
33da0859b9 Split academic year into seasons if the academic year's range is
initialized with a season, i.e. "2024-25 Fall".  Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly.   Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year.  So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring".  Add tests
for the new range format.  Set the default cutoff for the start of Spring season will be the last Sunday in February
2024-04-25 09:21:04 -07:00
5789ebf564 Faster admin data loader + rename School.school_hash 2024-04-22 14:46:37 -04:00
Nelson Jovel
37610e732f chore: alphabetize districts when writing filename for cleaned files 2024-02-23 11:27:59 -08:00
Nelson Jovel
0a32fb50ff fix: no longer support 'form' in filename when cleaning. Only look for 'part X' and add that to the filename if it exists 2024-02-22 10:55:48 -08:00
Nelson Jovel
e7fb009425 chore: refactor Gender out out of survey_item_values row 2023-12-20 11:08:23 -08:00
Nelson Jovel
24dbf33bcd chore: remove unneeded comment 2023-12-11 15:55:00 -08:00
Nelson Jovel
f028e6c884 feat: if the filename includes the words 'form' or 'part' add that to the resulting cleaned filename 2023-12-11 15:39:20 -08:00
Nelson Jovel
3db0f9f757 fix: Print out message to make clean when there are duplicate headers
present in the raw survey file
2023-12-07 12:28:24 -08:00
Nelson Jovel
e325f38c43 Convert gender and race text into qualtrics codes during cleaning. Abide by 'prefer not to disclose' for self reported race. Give priority to self reported data but use SIS information as backup 2023-12-06 14:10:16 -08:00
Nelson Jovel
8bebe7db42 chore: since it's now possible for there to be multiple district and dese id columns separated by a dash and a number, be more explicit when we only want to filter out survey item ids that end in a -1 2023-11-06 14:52:21 -08:00
rebuilt
e3fbbabce5 feat: We no longer trust the progress number that gets exported from qualtrics. Instead during the cleaning progress, perform a manual count of the number of responses to filter out rows that don't meet the minimum threshold. 2023-10-27 15:12:24 -07:00
rebuilt
48e795fcfb feat: add special education disaggregation 2023-10-06 11:41:52 -07:00
rebuilt
060d7aa55a Add disaggregation by ELL 2023-09-29 19:29:23 -07:00
rebuilt
714b90b3eb fix: ensure cleaner outputs columns for all survey items. Before the fix, if a survey item varient (ending in -1, ie s-tint-q1-1) did not have a matching survey item s-tint-q1, the resulting csv would not include that column 2023-08-23 15:30:47 -07:00
Nelson Jovel
0a2c5e02c5 feat: add ability to merge disaggregation data with raw survey data to
produce a cleaned csv with merged income disaggregation columns
2023-06-20 12:22:24 -07:00
rebuilt
896f0d9961 Don't write a file if there's an empty dataset 2023-06-07 12:38:44 -07:00
rebuilt
93d087a5de Use short district name for cleaned csv 2023-05-28 17:11:27 -07:00
rebuilt
4509c157fa Add automated data cleaning. Modify SurveyItemValues class to use regex
instead of hard coded values.  Produce a clean csv and a csv with all
the removed values and columns with reason for removal. Add script for
running cleaning for each project
2023-05-16 13:38:29 -07:00