gabehf/sqm-dashboards
1
0
Fork 0
mirror of https://github.com/edcommonwealth/sqm-dashboards.git synced 2026-03-11 08:21:12 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions

WIP: Update the parent table to hold a reference to a language

Browse source 
Update the data uploader script to read the language from the csv and update the language information for any parent items that already exist (or create database entries if none already exist)
This commit is contained in:
rebuilt 2025-04-15 15:03:32 -07:00
parent a48a2b1d7a
commit 446b3b1096
11 changed files with 182 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

45
spec/fixtures/demographic_glossary.csv vendored
View file

@ -1,21 +1,24 @@
ELL Value,ELL Type,ELL Headers,Sped Value,Sped Type,Sped Headers,Income Value,Income Type,Income Headers
EL student not 1st year,ELL,EL Student First Year,A,Special Education,Special Ed Status,Free Lunch,Economically Disadvantaged – Y,Low Income
"EL student, not 1st year",ELL,Raw ELL,active,Special Education,Raw SPED,Reduced Lunch,Economically Disadvantaged – Y,Raw Income
EL student 1st year,ELL,ELL- SIS,1,Special Education,SPED- SIS,LowIncome,Economically Disadvantaged – Y,SES- SIS
"EL student, 1st year",ELL,DirectCert,Special Education,Special Education,SPED,Low Income,Economically Disadvantaged – Y,EconDisadvantaged
EL - Early Child. or PK,ELL,ELL,Referred,Not Special Education,,Reduced price lunch,Economically Disadvantaged – Y,Income SIS
1,ELL,English Learner,Ineligible,Not Special Education,,TRUE,Economically Disadvantaged – Y,SES
lep student 1st year,ELL,,I,Not Special Education,,1,Economically Disadvantaged – Y,DirectCert
lep student not 1st year,ELL,,exited,Not Special Education,,Not Eligible,Economically Disadvantaged – N,
LEP Not1stYr,ELL,,0,Not Special Education,,FALSE,Economically Disadvantaged – N,
LEP1stYr US Sch,ELL,,Not Special Education,Not Special Education,,0,Economically Disadvantaged – N,
Does not apply,Not ELL,,Does not apply,Not Special Education,,[blanks],Economically Disadvantaged – N,
0,Not ELL,,[blanks],Not Special Education,,#NA,Unknown,
2,Not ELL,,#NA,Unknown,,NA,Unknown,
3,Not ELL,,NA,Unknown,,N/A,Unknown,
[blanks],Not ELL,,N/A,Unknown,,#N/A,Unknown,
#NA,Unknown,,#N/A,Unknown,,Income,Unknown,
NA,Unknown,,SPED,Unknown,,Yes,Economically Disadvantaged – Y,
N/A,Unknown,,No special needs,Not Special Education,,No,Economically Disadvantaged – N,
#N/A,Unknown,,,,,,,
ELL,ELL,,,,,,,
ELL Value,ELL Type,ELL Headers,Sped Value,Sped Type,Sped Headers,Income Value,Income Type,Income Headers,Language Value,Language Type
EL student not 1st year,ELL,EL Student First Year,A,Special Education,Special Ed Status,Free Lunch,Economically Disadvantaged – Y,Low Income,1,English
"EL student, not 1st year",ELL,Raw ELL,active,Special Education,Raw SPED,Reduced Lunch,Economically Disadvantaged – Y,Raw Income,10,English
EL student 1st year,ELL,ELL- SIS,1,Special Education,SPED- SIS,LowIncome,Economically Disadvantaged – Y,SES- SIS,2,Portuguese
"EL student, 1st year",ELL,DirectCert,Special Education,Special Education,SPED,Low Income,Economically Disadvantaged – Y,EconDisadvantaged,3,Spanish
EL - Early Child. or PK,ELL,ELL,Referred,Not Special Education,,Reduced price lunch,Economically Disadvantaged – Y,Income SIS,99,Unknown
1,ELL,English Learner,Ineligible,Not Special Education,,TRUE,Economically Disadvantaged – Y,SES ,100,Unknown
lep student 1st year,ELL,ELL Status- SIS,I,Not Special Education,,1,Economically Disadvantaged – Y,DirectCert,,
lep student not 1st year,ELL,,exited,Not Special Education,,Not Eligible,Economically Disadvantaged – N,,,
LEP Not1stYr,ELL,,0,Not Special Education,,FALSE,Economically Disadvantaged – N,,,
LEP1stYr US Sch,ELL,,Not Special Education,Not Special Education,,0,Economically Disadvantaged – N,,,
Does not apply,Not ELL,,Does not apply,Not Special Education,,[blanks],Economically Disadvantaged – N,,,
0,Not ELL,,[blanks],Not Special Education,,#NA,Unknown,,,
2,Not ELL,,#NA,Not Special Education,,NA,Unknown,,,
3,Not ELL,,NA,Not Special Education,,N/A,Unknown,,,
[blanks],Not ELL,,N/A,Not Special Education,,#N/A,Unknown,,,
#NA,Unknown,,#N/A,Not Special Education,,Income,Unknown,,,
NA,Unknown,,SPED,Unknown,,Yes,Economically Disadvantaged – Y,,,
N/A,Unknown,,No special needs,Not Special Education,,No,Economically Disadvantaged – N,,,
#N/A,Unknown,,Not SPED,Not Special Education,,,,,,
ELL,ELL,,,,,,,,,
LEP Not 1st Year,ELL,,,,,,,,,
Yes,ELL,,,,,,,,,
No,Not ELL,,,,,,,,,

1 ELL Value ELL Type ELL Headers Sped Value Sped Type Sped Headers Income Value Income Type Income Headers Language Value Language Type
2 EL student not 1st year ELL EL Student First Year A Special Education Special Ed Status Free Lunch Economically Disadvantaged – Y Low Income 1 English
3 EL student, not 1st year ELL Raw ELL active Special Education Raw SPED Reduced Lunch Economically Disadvantaged – Y Raw Income 10 English
4 EL student 1st year ELL ELL- SIS 1 Special Education SPED- SIS LowIncome Economically Disadvantaged – Y SES- SIS 2 Portuguese
5 EL student, 1st year ELL DirectCert Special Education Special Education SPED Low Income Economically Disadvantaged – Y EconDisadvantaged 3 Spanish
6 EL - Early Child. or PK ELL ELL Referred Not Special Education Reduced price lunch Economically Disadvantaged – Y Income SIS 99 Unknown
7 1 ELL English Learner Ineligible Not Special Education TRUE Economically Disadvantaged – Y SES 100 Unknown
8 lep student 1st year ELL ELL Status- SIS I Not Special Education 1 Economically Disadvantaged – Y DirectCert
9 lep student not 1st year ELL exited Not Special Education Not Eligible Economically Disadvantaged – N
10 LEP Not1stYr ELL 0 Not Special Education FALSE Economically Disadvantaged – N
11 LEP1stYr US Sch ELL Not Special Education Not Special Education 0 Economically Disadvantaged – N
12 Does not apply Not ELL Does not apply Not Special Education [blanks] Economically Disadvantaged – N
13 0 Not ELL [blanks] Not Special Education #NA Unknown
14 2 Not ELL #NA Unknown Not Special Education NA Unknown
15 3 Not ELL NA Unknown Not Special Education N/A Unknown
16 [blanks] Not ELL N/A Unknown Not Special Education #N/A Unknown
17 #NA Unknown #N/A Unknown Not Special Education Income Unknown
18 NA Unknown SPED Unknown Yes Economically Disadvantaged – Y
19 N/A Unknown No special needs Not Special Education No Economically Disadvantaged – N
20 #N/A Unknown Not SPED Not Special Education
21 ELL ELL
22 LEP Not 1st Year ELL
23 Yes ELL
24 No Not ELL

3
spec/services/cleaner_spec.rb
View file

@ -301,6 +301,7 @@ end
def reads_headers_from_raw_csv(processed_data)
processed_data in [headers, clean_csv, log_csv, data]
expect(headers.to_set.sort).to eq ["StartDate", "EndDate", "Status", "IPAddress", "Progress", "Duration (in seconds)",
"Finished", "RecordedDate", "ResponseId", "District", "School",
"LASID", "Gender", "Race", "What grade are you in?", "s-tint-q1",
@ -314,7 +315,7 @@ def reads_headers_from_raw_csv(processed_data)
"s-grit-q1", "s-grit-q2", "s-grit-q3", "s-grit-q4", "s-expa-q1", "s-poaf-q1", "s-poaf-q2", "s-poaf-q3",
"s-poaf-q4", "s-tint-q1-1", "s-tint-q2-1", "s-tint-q3-1", "s-tint-q4-1", "s-tint-q5-1", "s-acpr-q1-1",
"s-acpr-q2-1", "s-acpr-q3-1", "s-acpr-q4-1", "s-peff-q1-1", "s-peff-q2-1", "s-peff-q3-1", "s-peff-q4-1",
"s-peff-q5-1", "s-peff-q6-1", "Raw Income", "Income", "Raw ELL", "ELL", "Raw SpEd", "SpEd", "Progress Count"].to_set.sort
"s-peff-q5-1", "s-peff-q6-1", "Raw Income", "Income", "Raw ELL", "ELL", "Raw SpEd", "SpEd", "Progress Count", "Housing Status", "Raw Housing Status", "Home Language"].to_set.sort
end
def invalid_rows_are_rejected_for_the_correct_reasons(data)