Just change `MySuperSecretPassword` to the password you want to use with Postgres, and `~/postgres-data` to the folder where you want to store the database data.
(MacOS, Optional), you can use Homebrew:
Then, confirm it is running using `docker ps`
```
$ docker ps
│ CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
│ 761a4dddbbc0 postgres:13 "docker-entrypoint.s…" 24 minutes ago Up 22 minutes 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp sqm-postgres
```
#### (MacOS, Optional), you can use Homebrew:
```
brew install postgres
brew services start postgresql
```
Linux:
#### Linux:
Install postgres. Known working version is version 13
@ -48,14 +66,6 @@ bundle install
bundle exec rake db:create db:schema:load db:seed
```
Or if you only want to seed Lowell schools
```bash
bundle install
bundle exec rake db:create db:schema:load
bundle exec rake data:seed_only_lowell
```
Install the javascript dependencies
```bash
@ -76,10 +86,6 @@ The seed file populates the following tables
SurveyItemResponses does not get populated at this stage.
### Database
Postgres
### Gems
| Name | Description |
@ -105,14 +111,14 @@ Postgres
None yet. Hoping to integrate with Powerschool and Aspen for school administrative data.
### Javascript
### JavaScript
Esbuild is used as the javascript bundler. Scripts for esbuild are defined in package.json e.g. `yarn build`. This script will run if in development with `bin/dev`.
The javascript testing library is jest. Manually run test with `yarn test`. Javascript tests will also run with `bundle exec rake`.
Stimulus is installed. Create a stimulus controller with `./bin/rails generate stimulus [controller]`. If you create a stimulus controller manually, you can add it to `index.js` with the command `stimulus:manifest:update`.
### css
### CSS
Bootstrap 5
@ -123,10 +129,11 @@ SurveyItemResponses is the most important table to understand. SurveyItemRespons
Some notes:
- The data loading task assumes that the CSV files live in the `#{RAILS_ROOT}/data/survey_responses` directory
- The data loading task assumes that the CSV files live in SFTP servers whose connection strings are stored in the environment
- Data loading assumes the CSVs live in the the `/data/survey_responses/clean` directory. Usually, CSVs are actually stored in a further directory for each academic year.
- The data loading task is idempotent, i.e. it can be run multiple times without duplicating previously-ingested data
How to run the data loading task:
How to run the data loading task at the default directory:
```bash
# locally
@ -143,22 +150,16 @@ Or if you want to load data from a specific directory
$ heroku run:detached -a mciea-dashboard bundle exec rake data:load_admin_data
```
### Load Response Rates
### Load enrollment and staffing data
Many parts of the site rely on the response rate table being populated. The response rate is taken into account when determining if there is sufficient data to show
Enrollment and staffing numbers are taken from the DESE website.
To scrape staffing data from dese:
```bash
# locally
$ bundle exec rake data:reset_response_rates
# on heroku staging environment
$ heroku run:detached -a mciea-beta bundle exec rake data:reset_response_rates
# on heroku production environment
$ heroku run:detached -a mciea-dashboard bundle exec rake data:reset_response_rates
bundle exec rake scrape:staffing
```
### Load Student demographic information
Student demographic information is needed for the Analyze page disagregation
To scrape enrollment data from dese:
```bash
# locally
$ bundle exec rake data:load_students
# on heroku staging environment
$ heroku run:detached -a mciea-beta bundle exec rake data:load_students
# on heroku production environment
$ heroku run:detached -a mciea-dashboard bundle exec rake data:load_students
bundle exec rake scrape:enrollment
```
Or if you only want to load students for Lowell schools
Then to load it, run the seeder:
```bash
# locally
$ bundle exec rake data:load_students_for_lowell
bundle exec rake db:seed
```
### Load scores for each race
For performance, the scores for an individual race must be precalculated. Make sure to load response rates, and student demographic information before recaculating scores by race
Or load it remotely on heroku
```bash
heroku run:detached -a mciea-beta bundle exec rake db:seed
```
Or to load it for the lowell dashboard specifically
```bash
# locally
$ bundle exec rake data:reset_race_scores
bundle exec rake data:seed_only_lowell
```
# on heroku staging environment
$ heroku run:detached -a mciea-beta bundle exec rake data:reset_race_scores
# on heroku production environment
$ heroku run:detached -a mciea-dashboard bundle exec rake data:reset_race_scores
### Upload cleaned data to SFTP
You can upload cleaned lowell data to the SFTP server with
