gabehf/sqm-dashboards
1
0
Fork 0
mirror of https://github.com/edcommonwealth/sqm-dashboards.git synced 2026-03-07 13:38:18 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions

scrape admin, enrollments, and staffing data

Browse source
This commit is contained in:
rebuilt 2026-03-05 12:17:44 -08:00
parent b0d4f19bba
commit ddd20b2053
11 changed files with 38608 additions and 3629 deletions
Download patch file Download diff file Expand all files Collapse all files

2
app/services/dese/scraper.rb
View file

@ -1,6 +1,6 @@
module Dese
module Scraper
DELAY = 20 # The dese site will block you if you hit it too many times in a short period of time
DELAY = 4 # The dese site will block you if you hit it too many times in a short period of time
Prerequisites = Struct.new("Prerequisites", :filepath, :url, :selectors, :submit_id, :admin_data_item_id,
:calculation)