118 Commits (170bc79f96f09a4a6c4594dcf839a384adb0a697)

Author	SHA1	Message	Date
rebuilt	359e266a6c	Remove unused TODOs	3 years ago
rebuilt	3f2a7dff50	Fix problem with dese scraper lumping in 2021-22 data as 2022-23 data. ... Deleted unused csvs. Turned off puts statements in admin loader. Remove old, now unused admin data loader class.	3 years ago
rebuilt	4b7d2e6feb	Only return files in sftp directory, not other directories	3 years ago
rebuilt	f68b650d8b	Calculate response rate on the fly instead of looking it up from the db ... when calculating response rates.	3 years ago
rebuilt	10cedd2310	Remove unused filename	3 years ago
rebuilt	5b2bae0d92	Fix ThreeATwo scraper	3 years ago
rebuilt	128748addd	Update logic for calculating student response rate. Remove references ... to survey table. We no longer check or keep track of the survey type. Instead we look in the database to see if a survey item has at least 10 responses. If it does, that survey item was presented to the respondent and we count it, and all responses when calculating the response rate. Remove response rate timestamp from caching logic because we no longer add the response rate to the database. All response rates are calculated on the fly Update three_b_two scraper to use teacher only numbers swap over to using https://profiles.doe.mass.edu/statereport/gradesubjectstaffing.aspx as the source of staffing information	3 years ago
rebuilt	b250ebe415	Memoize schools in SurveyItemValues and academic_years in AcademicYear ... for performace improvement	3 years ago
rebuilt	c15cb7b483	Change survey data loader spec to use factorybot objects instead of loading seeds. Change databasecleaner to use transaction. Add back babel-preset dependency to fix failing javascript test in production.	3 years ago
rebuilt	db3e4f1178	Set config variable for SFTP_TO_GO server	3 years ago
rebuilt	5e88c8be1b	Transition from Lowell to ECP. Update text and links to reflect the new Organization.	3 years ago
rebuilt	4c4ccc01cc	Merge branch 'rpp-response-rate' to bring in changes to test files	3 years ago
rebuilt	6b31fa9115	Batch imports for staffing data	3 years ago
rebuilt	d059177f0c	load total students and batch importing records	3 years ago
rebuilt	2362d884eb	Convert admin data loader from using seeder to using factory	3 years ago
rebuilt	d0219217de	Convert response rate loader spec from using the seeder to using the factory	3 years ago
rebuilt	06f9d2f0e9	Scrape enrollment and staffing information. Seed enrollment and staffing information. Update DatabaseCleaner so it cleans up leftover information in the database. Remove old admin csvs from codebase.	3 years ago
rebuilt	c0332955f3	move csv require statement to application.rb	3 years ago
rebuilt	ae6d7151b3	Load student respondent information from enrollment loader	3 years ago
rebuilt	984fedff02	Add academic year column to enrollment file. Add enrollment loader to seeder file	3 years ago
rebuilt	52d2676f80	Populate db with grade level data	3 years ago
rebuilt	69c089d9bb	Remove occurances of MCIEA	3 years ago
rebuilt	01851d8527	Make sure blank grades in csv load as nils instead of zeros	3 years ago
rebuilt	3a744db975	Start adding kindergarten	3 years ago
rebuilt	c6b8518cb1	Fix bug with not all survey responses loading when using sftp loader	3 years ago
rebuilt	b1200dca33	update default folder for survey responses	3 years ago
rebuilt	7a159f7ddf	Process 1000 rows at a time to limit memory usage in production	3 years ago
rebuilt	79adff5faa	Don't print sftptogo_url	3 years ago
rebuilt	03c6bff7bb	Load survey responses from sftp folder	3 years ago
rebuilt	1426e7cc63	add proof of concept sftp downloader	3 years ago
rebuilt	eeeb4f5d14	Add Stoklosa survey responses	3 years ago
rebuilt	d5533bb756	Support survey item variants. Survey item id's in the format s-sbel-q1-1 will load into s-sbel-q1	3 years ago
rebuilt	ed421296c7	Fix bug that caused an incorrect score of 1 to be assigned for admin ... data item related to parity index of teachers of color to students of color. Now if there's not enough data to calculate a score, it produces 'NA' instead of 1. Also added admin data for a-reso-i1 and a-curv-i1	3 years ago
rebuilt	7b9abe3793	Scrape latest data from DESE	3 years ago
rebuilt	850ef9d1cd	Make sure demographic loader doesn't break when changing the designation of a Race	3 years ago
rebuilt	53a08d41bf	Change label for unknown race	3 years ago
rebuilt	a1b7db9b2e	Add ability to load student demographic information for only lowell	3 years ago
rebuilt	d49cf918ac	Add a way to seed only lowell schools	3 years ago
rebuilt	fc9e73f656	make sure survey responses for Butler get loaded correctly	3 years ago
rebuilt	9bbba64e9e	Add rules to the survey data loader to allow skipping non-lowell schools	3 years ago
Nelson Jovel	43bc55ed32	Query for survey items once instead of on each row	3 years ago
rebuilt	83fa255e5d	Make utility method private	3 years ago
rebuilt	3ca270e3b1	Fix more malformed csv errors	3 years ago
rebuilt	360579852c	Send fewer sql queries when loading survey responses	3 years ago
rebuilt	9ae94a2dae	further reduce batch size to avoid exceeding memory quota	3 years ago
rebuilt	bb472ce6c3	Reduce number of rows to process to avoid exceeding memory limit on heroku	3 years ago
rebuilt	4626d43e3e	Add changes to demographic loader so it the tests don't fail when run in parallel	3 years ago
rebuilt	7d7bee6498	Add gender disagreggation	3 years ago
rebuilt	a1f3dfaa33	Load gender information to survey item responses	3 years ago
rebuilt	8cdd1b6285	Create gender class. Load genders to database from demographics file	3 years ago