Just change `MySuperSecretPassword` to the password you want to use with Postgres, and `~/postgres-data` to the folder where you want to store the database data.
(MacOS, Optional), you can use Homebrew:
Then, confirm it is running using `docker ps`
```
$ docker ps
│ CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
│ 761a4dddbbc0 postgres:13 "docker-entrypoint.s…" 24 minutes ago Up 22 minutes 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp sqm-postgres
```
#### (MacOS, Optional), you can use Homebrew:
```
```
brew install postgres
brew install postgres
brew services start postgresql
brew services start postgresql
```
```
Linux:
#### Linux:
Install postgres. Known working version is version 13
Install postgres. Known working version is version 13
@ -48,14 +66,6 @@ bundle install
bundle exec rake db:create db:schema:load db:seed
bundle exec rake db:create db:schema:load db:seed
```
```
Or if you only want to seed Lowell schools
```bash
bundle install
bundle exec rake db:create db:schema:load
bundle exec rake data:seed_only_lowell
```
Install the javascript dependencies
Install the javascript dependencies
```bash
```bash
@ -76,10 +86,6 @@ The seed file populates the following tables
SurveyItemResponses does not get populated at this stage.
SurveyItemResponses does not get populated at this stage.
### Database
Postgres
### Gems
### Gems
| Name | Description |
| Name | Description |
@ -105,14 +111,14 @@ Postgres
None yet. Hoping to integrate with Powerschool and Aspen for school administrative data.
None yet. Hoping to integrate with Powerschool and Aspen for school administrative data.
### Javascript
### JavaScript
Esbuild is used as the javascript bundler. Scripts for esbuild are defined in package.json e.g. `yarn build`. This script will run if in development with `bin/dev`.
Esbuild is used as the javascript bundler. Scripts for esbuild are defined in package.json e.g. `yarn build`. This script will run if in development with `bin/dev`.
The javascript testing library is jest. Manually run test with `yarn test`. Javascript tests will also run with `bundle exec rake`.
The javascript testing library is jest. Manually run test with `yarn test`. Javascript tests will also run with `bundle exec rake`.
Stimulus is installed. Create a stimulus controller with `./bin/rails generate stimulus [controller]`. If you create a stimulus controller manually, you can add it to `index.js` with the command `stimulus:manifest:update`.
Stimulus is installed. Create a stimulus controller with `./bin/rails generate stimulus [controller]`. If you create a stimulus controller manually, you can add it to `index.js` with the command `stimulus:manifest:update`.
### css
### CSS
Bootstrap 5
Bootstrap 5
@ -123,10 +129,11 @@ SurveyItemResponses is the most important table to understand. SurveyItemRespons
Some notes:
Some notes:
- The data loading task assumes that the CSV files live in the `#{RAILS_ROOT}/data/survey_responses` directory
- The data loading task assumes that the CSV files live in SFTP servers whose connection strings are stored in the environment
- Data loading assumes the CSVs live in the the `/data/survey_responses/clean` directory. Usually, CSVs are actually stored in a further directory for each academic year.
- The data loading task is idempotent, i.e. it can be run multiple times without duplicating previously-ingested data
- The data loading task is idempotent, i.e. it can be run multiple times without duplicating previously-ingested data
How to run the data loading task:
How to run the data loading task at the default directory:
```bash
```bash
# locally
# locally
@ -143,22 +150,16 @@ Or if you want to load data from a specific directory
$ heroku run:detached -a mciea-dashboard bundle exec rake data:load_admin_data
$ heroku run:detached -a mciea-dashboard bundle exec rake data:load_admin_data
```
```
### Load Response Rates
### Load enrollment and staffing data
Many parts of the site rely on the response rate table being populated. The response rate is taken into account when determining if there is sufficient data to show
Enrollment and staffing numbers are taken from the DESE website.
To scrape staffing data from dese:
```bash
```bash
# locally
bundle exec rake scrape:staffing
$ bundle exec rake data:reset_response_rates
# on heroku staging environment
$ heroku run:detached -a mciea-beta bundle exec rake data:reset_response_rates
# on heroku production environment
$ heroku run:detached -a mciea-dashboard bundle exec rake data:reset_response_rates
```
```
### Load Student demographic information
To scrape enrollment data from dese:
Student demographic information is needed for the Analyze page disagregation
```bash
```bash
# locally
bundle exec rake scrape:enrollment
$ bundle exec rake data:load_students
# on heroku staging environment
$ heroku run:detached -a mciea-beta bundle exec rake data:load_students
# on heroku production environment
$ heroku run:detached -a mciea-dashboard bundle exec rake data:load_students
```
```
Then to load it, run the seeder:
Or if you only want to load students for Lowell schools
```bash
```bash
# locally
bundle exec rake db:seed
$ bundle exec rake data:load_students_for_lowell
```
```
### Load scores for each race
Or load it remotely on heroku
For performance, the scores for an individual race must be precalculated. Make sure to load response rates, and student demographic information before recaculating scores by race
```bash
heroku run:detached -a mciea-beta bundle exec rake db:seed
```
Or to load it for the lowell dashboard specifically
```bash
```bash
# locally
bundle exec rake data:seed_only_lowell
$ bundle exec rake data:reset_race_scores
```
# on heroku staging environment
$ heroku run:detached -a mciea-beta bundle exec rake data:reset_race_scores
# on heroku production environment
### Upload cleaned data to SFTP
$ heroku run:detached -a mciea-dashboard bundle exec rake data:reset_race_scores
You can upload cleaned lowell data to the SFTP server with
