051f156126
chore: add parsing rules for income: 'yes', 'no' and sped: 'No special needs' and ell: 'ELL'
2 years ago
74fdc89fc2
chore: update glossary values to include parsing rules for when we encounter headers
2 years ago
c4d4c35766
update parsing rules from glossary
2 years ago
8e7fbdfb2c
add disaggregation glossary
2 years ago
0a27538747
chore: add a test for categorizing sped values of 1 and 0 as 'Special Education' and 'Not Special Education'
2 years ago
65c25fc3c7
Add ell income and sped parsing rules for SIS data. Add tests for the
...
new inputs.
2 years ago
2561fa28fc
feat: Split academic year into seasons if the academic year's range is
...
initialized with a season, i.e. "2024-25 Fall". Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly. Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year. So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring". Add tests
for the new range format. Set the default cutoff for the start of Spring season will be the last Sunday in February
2 years ago
67ffc996a8
Revert "Split academic year into seasons if the academic year's range is"
...
This reverts commit a5d4cccb37
2 years ago
a5d4cccb37
Split academic year into seasons if the academic year's range is
...
initialized with a season, i.e. "2024-25 Fall". Update scapers for
admin data, enrollment and staffing to use the new range standard
correctly. Update the loaders for admin data, enrollment and staffing
so that it populates all seasons in a given year. So admin data for
2024-25 gets loaded into "2024-25 Fall" and "2024-25 Spring". Add tests
for the new range format. Set the default cutoff for the start of Spring season will be the last Sunday in February
2 years ago
Gabe Farrell
6fac759ec2
Faster admin data loader + rename School.school_hash
2 years ago
993fafcc0e
Revert "fix: Makes visible the graphs by grade"
...
This reverts commit 455f716b6d
2 years ago
455f716b6d
fix: Makes visible the graphs by grade
2 years ago
e17a029366
fix: fix broken tests
2 years ago
9bfb76db5a
match an additional format for Dates. Supported dates are now '1/10/2022 14:21:45' '2022-1-10T14:21:45' '2022-1-10 14:21:45'
2 years ago
fc2c52984c
fix: correct recorded dates of sample csvs
2 years ago
c3cb05701f
feat: Support two date formats: ISO 8601 and the standard US date format
...
used in google sheets
2 years ago
aa7af11a4e
fix: fix failing test
2 years ago
cc8ed48204
fix: no longer support 'form' in filename when cleaning. Only look for 'part X' and add that to the filename if it exists
2 years ago
d907e2742e
fix: fix failing tests
2 years ago
59b7416a3a
feat: change 'school admin data' to 'school data'
2 years ago
c565dbe46f
chore: remove lowell seeding rules
2 years ago
a0c0b1d01d
chore: reenable test spec that tests data loader for races
2 years ago
d4109fda6f
chore: refactor Race out of survey_item_values
2 years ago
3e4ef9cb08
chore: refactor Gender out out of survey_item_values row
2 years ago
6e7e64c039
chore: rename counts_by_grade to enrollment_by_grade for clarity
2 years ago
6a24d4fa23
chore: Make sure 'hispanic' column only gets applied when using SIS race information
2 years ago
ce43f52bd5
feat: if the filename includes the words 'form' or 'part' add that to the resulting cleaned filename
2 years ago
a15b01a3e1
fix: instead of looking for 'asian' at the start of a word, look for it
...
after a word boundary. This means it still doesn't get confused with
caucasian and it's more flexible whan asian appears inside other text
such as 'Caucasian and Asian and Black'
2 years ago
8a0ba0dbea
chore: various fixes for race and gender categorization during cleaning.
...
Also add tests for race and gender categorization
2 years ago
2ef24caf70
Lower threshold for the number of valid student responses from 17 to 11
2 years ago
b983f1e144
chore: fix categorization of gender
2 years ago
f27a590c5a
Convert gender and race text into qualtrics codes during cleaning. Abide by 'prefer not to disclose' for self reported race. Give priority to self reported data but use SIS information as backup
2 years ago
97ddb09167
chore: add test for checking duplicate headers during cleaning process
2 years ago
6d84204f83
Add race and gender columns to cleaned cvs files when those headers are
...
missing
2 years ago
a3f9e46414
chore: when searching for dese id, split up pattern so that to be more explicit about the order in which to search out the columns that might have the dese ID we're looking for.
2 years ago
019b954ffa
feat: load student responses in the same pass as loading the survey responses
...
chore: remove student loader since loading students is now done with the survey response loader
2 years ago
9110be31c6
chore: fix failing test
2 years ago
b2fdbe5756
feat: We no longer trust the progress number that gets exported from qualtrics. Instead during the cleaning progress, perform a manual count of the number of responses to filter out rows that don't meet the minimum threshold.
2 years ago
e45a4f96dd
last commit
2 years ago
ef44c41965
feat: add special education disaggregation
2 years ago
18ab51c860
chore: upgrade to rails 7.1.
...
upgrade rspec
fix failing tests
upgrade devise
2 years ago
2fd56047d4
Add disaggregation by ELL
2 years ago
490522eb1e
feat: support multiple columns for race and gender information
2 years ago
7bd7923d41
fix: ensure cleaner outputs columns for all survey items. Before the fix, if a survey item varient (ending in -1, ie s-tint-q1-1) did not have a matching survey item s-tint-q1, the resulting csv would not include that column
2 years ago
e58ea3f1af
feat: Round up response rate. Survey response rates above 24.5 will now meet sufficiency
2 years ago
8cf9b4eeb8
Add test for income group
2 years ago
a97838c194
Reorder gender columns
2 years ago
2ac30bb107
feat: Add income table to the database. Add seeder for income. Add a reference to income from survey item response. Update the loader to import income data from the survey response csv. Refactor analyze controller to extract presenter. Add corresponding specs. Add income graph to analyze page
2 years ago
f230f76bc6
fix: Overall response rate was incorrectly using the updated_at date instead of the recorded date. Also, it was just using the last date for all academic years instead of the last date the survey was taken per academic year.
2 years ago
a4332f6a05
chore: remove outdated admin data loader file. We now use Dese::Loader to load school level data
2 years ago
Nelson Jovel
Gabe Farrell