gabehf/sqm-dashboards
1
0
Fork 0
mirror of https://github.com/edcommonwealth/sqm-dashboards.git synced 2026-03-07 21:48:16 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update student survey responses for the 2018-19 school year

Browse source
This commit is contained in:
Nelson Jovel 2022-04-11 10:31:11 -07:00
parent fe319a51b3
commit a8507d817c
4 changed files with 26 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -141,6 +141,11 @@ $ ./scripts/load_survey_responses_on_heroku beta
$ ./scripts/load_survey_responses_on_heroku dashboard
```
There is also an example one-off task to load a single csv at a time.
```bash
bundle exec rake one_off:load_2018_19_student_responses
```
### Loading Admin Data Values
Loading admin data is similar to loading survey item responses. Run the one of the following scripts to load admin data to a selected environment.

2
data/survey_responses/2018-19_student_survey_responses.csv
View file

@ -1,4 +1,4 @@
StartDate,Response.Type,IPAddress,Progress,Duration..in.seconds.,Finished,RecordedDate,ResponseID,Location.Latitude,DistributionChannel,UserLanguage,district,student_id,grade,s.emsa.q1,s.emsa.q2,s.emsa.q3,s.tint.q1,s.tint.q2,s.tint.q3,s.tint.q4,s.tint.q5,s-acpr-q1,s-acpr-q2,s-acpr-q3,s-acpr-q4,s-cure-q1,s-cure-q2,s-cure-q3,s-cure-q4,s-sten-q1,s-sten-q2,s-sten-q3,s-sper-q1,s-sper-q2,s-sper-q3,s-sper-q4,s-civp-q1,s-civp-q2,s-civp-q3,s-civp-q4,s-grmi-q1,s-grmi-q2,s-grmi-q3,s-grmi-q4,s-appa-q1,s-appa-q2,s-appa-q3,s-peff-q1,s-peff-q2,s-peff-q3,s-peff-q4,s-peff-q5,s-peff-q6,s-sbel-q1,s-sbel-q2,s-sbel-q3,s-sbel-q4,s-sbel-q5,s-phys-q1,s-phys-q2,s-phys-q3,s-phys-q4,s-vale-q1,s-vale-q2,s-vale-q3,s-vale-q4,s-acst-q1,s-acst-q2,s-acst-q3,s-sust-q1,s-sust-q2,s-grit-q1,s-grit-q2,s-grit-q3,s-grit-q4,s-expa-q1,s-poaf-q1,s-poaf-q2,s-poaf-q3,s-poaf-q4,gender,Q90_3_TEXT,race,Q91_7_TEXT,language,MathTeacher,ScienceTeacher,SocialTeacher,EnglishTeacher,DESE ID
StartDate,Response.Type,IPAddress,Progress,Duration..in.seconds.,Finished,RecordedDate,ResponseID,Location.Latitude,DistributionChannel,UserLanguage,district,student_id,grade,s-emsa-q1,s-emsa-q2,s-emsa-q3,s-tint-q1,s-tint-q2,s-tint-q3,s-tint-q4,s-tint-q5,s-acpr-q1,s-acpr-q2,s-acpr-q3,s-acpr-q4,s-cure-q1,s-cure-q2,s-cure-q3,s-cure-q4,s-sten-q1,s-sten-q2,s-sten-q3,s-sper-q1,s-sper-q2,s-sper-q3,s-sper-q4,s-civp-q1,s-civp-q2,s-civp-q3,s-civp-q4,s-grmi-q1,s-grmi-q2,s-grmi-q3,s-grmi-q4,s-appa-q1,s-appa-q2,s-appa-q3,s-peff-q1,s-peff-q2,s-peff-q3,s-peff-q4,s-peff-q5,s-peff-q6,s-sbel-q1,s-sbel-q2,s-sbel-q3,s-sbel-q4,s-sbel-q5,s-phys-q1,s-phys-q2,s-phys-q3,s-phys-q4,s-vale-q1,s-vale-q2,s-vale-q3,s-vale-q4,s-acst-q1,s-acst-q2,s-acst-q3,s-sust-q1,s-sust-q2,s-grit-q1,s-grit-q2,s-grit-q3,s-grit-q4,s-expa-q1,s-poaf-q1,s-poaf-q2,s-poaf-q3,s-poaf-q4,gender,Q90_3_TEXT,race,Q91_7_TEXT,language,MathTeacher,ScienceTeacher,SocialTeacher,EnglishTeacher,DESE ID
3/12/19 8:47,IP Address,209.80.140.129,100,323,TRUE,2019-03-12T8:52:0,R_24IvMAxf0EV3C69,42.4125061,anonymous,EN,Revere,36434,4,NA,NA,NA,NA,NA,NA,NA,NA,3,2,3,2,NA,NA,NA,NA,NA,NA,NA,2,1,3,4,NA,NA,NA,NA,NA,NA,NA,NA,1,4,5,2,5,3,4,2,2,3,4,3,4,4,3,5,1,1,4,5,5,5,3,4,1,3,NA,4,1,5,1,5,1,4,2,4,2,,7,,EN,,,,,02480003
3/12/19 8:47,IP Address,209.80.140.129,100,530,TRUE,2019-03-12T8:55:0,R_1mWRko0fMVAqNRE,42.4125061,anonymous,EN,Revere,37788,4,NA,NA,NA,NA,NA,NA,NA,NA,5,4,5,5,NA,NA,NA,NA,NA,NA,NA,5,5,3,5,NA,NA,NA,NA,NA,NA,NA,NA,1,5,5,4,4,5,3,3,5,2,3,3,3,3,5,5,4,5,3,1,3,4,1,2,3,5,3,2,2,2,3,4,3,2,3,4,1,,6,,EN,,,,,02480003
3/12/19 8:47,IP Address,209.80.140.129,100,558,TRUE,2019-03-12T8:56:0,R_3NVwGDvFnTiR2rf,42.4125061,anonymous,EN,Revere,37810,4,5,4,2,5,4,5,2,2,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,1,1,3,3,NA,NA,NA,4,4,2,3,4,5,4,1,1,4,4,5,5,3,2,1,4,2,2,NA,NA,2,1,3,5,4,4,5,3,4,4,3,5,2,,7,,EN,,,,,02480003

Can't render this file because it is too large.

17
doc/architectural_decision_records/6.md Normal file
View file

@ -0,0 +1,17 @@
# Decision record 6
# Upload a single csv at a time
## Status
Completed
## Context
Uploading survey results takes a long time. It's on the order of 20-30 minutes per csv file.
## Decision
Instead of uploading all records each time, upload only a single csv file
## Consequences
There is a lot less time spent performing work that has already been done. This script, like the script to import all csvs, does not remove any responses. It only adds responses that do not already exist and modifies the likert value to the one found in the csv if the value differs from the database.

6
lib/tasks/one_off.rake
View file

@ -45,9 +45,9 @@ namespace :one_off do
measure_4aii.update! measure_id: '4A-i'
end
desc 'load boston 2018-19 responses'
task load_boston_2018_19_responses: :environment do
filepath = Rails.root.join('data', 'survey_responses', '2018-19_boston_student_survey_responses.csv')
desc 'load 2018-19 student responses'
task load_2018_19_student_responses: :environment do
filepath = Rails.root.join('data', 'survey_responses', '2018-19_student_survey_responses.csv')
puts "=====================> Loading data from csv at path: #{filepath}"
SurveyResponsesDataLoader.load_data filepath: filepath
puts "=====================> Completed loading #{SurveyItemResponse.count} survey responses"